79
Light and Sound, Light and Sound, Seeing and Listening Seeing and Listening 2.1 Perception 2.1 Perception 2.2 Light 2.2 Light 2.3 Color and Color Sensation 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.4 Sight and Visual Communication 2.5 Sound 2.5 Sound 2.6 Listening and Auditory Communication 2.6 Listening and Auditory Communication hapter 2 Theoretical Foundations of Multimedi

Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Embed Size (px)

Citation preview

Page 1: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Light and Sound, Light and Sound, Seeing and ListeningSeeing and Listening

2.1 Perception2.1 Perception

2.2 Light2.2 Light

2.3 Color and Color Sensation2.3 Color and Color Sensation

2.4 Sight and Visual Communication2.4 Sight and Visual Communication

2.5 Sound2.5 Sound

2.6 Listening and Auditory Communication2.6 Listening and Auditory Communication

Chapter 2 Theoretical Foundations of Multimedia

Page 2: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Perception and CognitionPerception and Cognition

Perception — the consciousness or awareness of objects Perception — the consciousness or awareness of objects or other data through the medium of the senses.or other data through the medium of the senses.

Cognition — high-order mental processes, such as Cognition — high-order mental processes, such as complex representation, inference, and interpretation, complex representation, inference, and interpretation, that lead to perception and then to understanding.that lead to perception and then to understanding.

Chapter 2 Theoretical Foundations of Multimedia

Page 3: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

““Regions of Interest” Regions of Interest” for Studying Perceptionfor Studying Perception

• The EnvironmentThe Environment• Incoming stimuliIncoming stimuli• Sensory surfaces and peripheral neuronsSensory surfaces and peripheral neurons• The BrainThe Brain• Effector systemsEffector systems• Motor responsesMotor responses

Chapter 2 Theoretical Foundations of Multimedia

Page 4: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Optical IllusionsOptical Illusions

Chapter 2 Theoretical Foundations of Multimedia

Müller-Lyer IllusionMüller-Lyer Illusion

Subjective contoursSubjective contours Archimedes spiralArchimedes spiral

Page 5: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Auditory IllusionsAuditory Illusions

• Bach sonatas and partitasBach sonatas and partitas• Ravel’s Ravel’s BoleroBolero

Chapter 2 Theoretical Foundations of Multimedia

Page 6: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Nature of LightThe Nature of Light

• A form of energy A form of energy • Particle nature of lightParticle nature of light• Waveform nature of lightWaveform nature of light• Early Philosophers and Researchers:Early Philosophers and Researchers:

PlatoPlato Sir Isaac Newton Sir Isaac Newton AlhazenAlhazen

Thomas Young Thomas Young Christopher ScheinerChristopher Scheiner Max Planck Max Planck

Chapter 2 Theoretical Foundations of Multimedia

Page 7: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

A Sine Wave

Page 8: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

1 sec. 2 sec. 3 sec.

One cycle per second, or 1 hertz (Hz)

Page 9: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

Two waves of equal frequency and different amplitudes

Page 10: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

Two waves and the wave resulting from their interaction

Page 11: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

Two equivalent waves in phasecancellation

Page 12: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

Two equivalent waves in phaseand the resultant wave

Page 13: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Behavior of WavesThe Behavior of Waves

Chapter 2 Theoretical Foundations of Multimedia

A fundamental waveform and its second and third harmonics

Page 14: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Elecromagnetic The Elecromagnetic SpectrumSpectrum

Chapter 2 Theoretical Foundations of Multimedia

Page 15: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Elecromagnetic The Elecromagnetic SpectrumSpectrum

Chapter 2 Theoretical Foundations of Multimedia

Page 16: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Speed of LightThe Speed of Light

Chapter 2 Theoretical Foundations of Multimedia

• 186,282.3959 miles per second

• Events within sight appear to be instantaneous

• Speed varies in different media

• Some materials absorb some ofthe light’s energy

Page 17: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

The Measurement of The Measurement of LightLight

Chapter 2 Theoretical Foundations of Multimedia

• Intensity or “strength” of radiation

• Radiance — the total amount of energy(light and heat) measured in watts

• Luminance — light strength perceivedby a human eye

• Brightness — subjective measure of how bright an object appears to a human

• Strength diminishes by inverse squareproportion to distance from source

Page 18: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

ReflectionReflection

Chapter 2 Theoretical Foundations of Multimedia

Reflection — light strikes a surface and the waves bounce off

• May cause problems on a monitor under the wrong lighting conditions

• Allows fiber optic cables to functionby causing the light being sentalong the fiber to “bounce” off ofthe walls of the cable and continue toward the other end

Page 19: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

RefractionRefraction

Chapter 2 Theoretical Foundations of Multimedia

Refraction — light passes through a different medium and has its direction changed; sometimes called the “bending” of light

Page 20: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

RefractionRefraction

Chapter 2 Theoretical Foundations of Multimedia

Prism separating white light

Page 21: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

RefractionRefraction

Chapter 2 Theoretical Foundations of Multimedia

Lenses focus or spread light

Page 22: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

ColorColor

Chapter 2 Theoretical Foundations of Multimedia

The sensation registered in the brain

when different wavelengths of light

impinge upon the retina of the eye,

causing a message to be sent along

the optic nerve to excite neurons in

various portions of the brain.

Page 23: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

ColorColor

Chapter 2 Theoretical Foundations of Multimedia

Reflected color occurs when a light illuminates an object and some of thewavelengths of the light are reflectedoff of the object, while others areabsorbed by it.

Transmitted or emitted color results from an atom or a molecule emittinglight energy of a characteristic wave-length after being excited.

Page 24: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Measuring ColorMeasuring Color

Chapter 2 Theoretical Foundations of Multimedia

Emission and Remission Curves

Page 25: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Measuring Color Measuring Color

• Chroma — hue, or a color’s pure expression• Saturation — the amount of pure color (not black or

white) in a chroma• Value — subtle difference between shades and tints• Lightness — amount of light reflected from an object• Brightness — luminance of an object in the context of its

surroundings

Chapter 2 Theoretical Foundations of Multimedia

Page 26: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Trichromatic Vision Trichromatic Vision

• Subtractive principle of color — reflected color• Additive principle of color — emitted color• Thomas Young — proposed that the eye is composed of

three different light-sensitive materials• Hermann von Helmholtz — identified the retinal cells for

the three colors• Young-Helmholtz Theory — three–component or

trichromatic vision theory

Chapter 2 Theoretical Foundations of Multimedia

Page 27: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Perception of ColorPerception of Color

• The eye sees different wavelengths and intensities very The eye sees different wavelengths and intensities very differentlydifferently

• VioletViolet is barely perceived is barely perceived• RedRed is seen very well is seen very well• The cornea is The cornea is yellowyellow, and absorbs , and absorbs greengreen, , blueblue, and , and VioletViolet• As people age, the cornea becomes even more As people age, the cornea becomes even more yellowyellow so so

they see even less they see even less greengreen, , blueblue, and , and VioletViolet

Chapter 2 Theoretical Foundations of Multimedia

Page 28: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Psychological Aspects of Psychological Aspects of ColorColor

• Descriptive phrases — seeing Descriptive phrases — seeing RedRed; feeling ; feeling blue; blue; a black a black mood; a mood; a yellow yellow coward coward

• Color is highly subjectiveColor is highly subjective• Colors are associated with emotionsColors are associated with emotions• Warm colors (Warm colors (RedRed and and yellowyellow) appear near) appear near• Cool colors (Cool colors (blueblue and and greengreen) appear more distant) appear more distant• Pastels appear relaxed; dark colors seem more angryPastels appear relaxed; dark colors seem more angry

Chapter 2 Theoretical Foundations of Multimedia

Page 29: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Color WheelColor Wheel

Chapter 2 Theoretical Foundations of Multimedia

blueblue

magentamagenta

redred

yellowyellow

greengreen

cyancyan

Page 30: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Colors on Computer Colors on Computer MonitorsMonitors

• Radiated vs. reflected light (recall the Radiated vs. reflected light (recall the additive additive principle vs. the subtractive principle vs. the subtractive principle of light)principle of light)

• RGB system — RGB system — Red, Red, green, green, blueblue• CYM system — CYM system — CyanCyan, , yellowyellow, , magentamagenta• HSV system — hue, saturation, & valueHSV system — hue, saturation, & value• HLS system — hue, lightness, & saturationHLS system — hue, lightness, & saturation• CIE system — Commission Internationale CIE system — Commission Internationale de l’Eclairagede l’Eclairage

Chapter 2 Theoretical Foundations of Multimedia

Page 31: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

RGB Color CubeRGB Color Cube

Chapter 2 Theoretical Foundations of Multimedia

greengreen

GG

yellowyellow

blueblue

redredRR

BB

black

gray linegray line

magentamagenta

cyancyanwhitewhite

Page 32: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Sight and Visual Sight and Visual CommunicationCommunication

• Physical process Physical process — light being trans-— light being trans- mitted and reflectedmitted and reflected• Physiological process Physiological process — light enters the — light enters the eye, falls on the eye, falls on the

retina, and is trans-retina, and is trans- mitted by the optic nervemitted by the optic nerve• Neurological process Neurological process — the brain reacts — the brain reacts to the impulses to the impulses

from the optic nervefrom the optic nerve• Psychological process Psychological process — the mind — the mind interprets the interprets the

information in the braininformation in the brain

Chapter 2 Theoretical Foundations of Multimedia

Page 33: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Sight and Visual Sight and Visual CommunicationCommunication

Chapter 2 Theoretical Foundations of Multimedia

The Camera ObscuraThe Camera Obscura

Page 34: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Sight and Visual Sight and Visual CommunicationCommunication

Chapter 2 Theoretical Foundations of Multimedia

Comparing Comparing a camera a camera and the and the human eyehuman eye

Page 35: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Aspects of Visual LiteracyAspects of Visual Literacy

• A prerequisite for the comprehension of A prerequisite for the comprehension of visual mediavisual media

• General cognitive consequencesGeneral cognitive consequences• Awareness of visual manipulationAwareness of visual manipulation• Aesthetic appreciationAesthetic appreciation

Chapter 2 Theoretical Foundations of Multimedia

Page 36: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Images and MeaningImages and Meaning

• SenseSense — the external reality denoted by — the external reality denoted by an imagean image• FeelingFeeling — the attitude conveyed by an — the attitude conveyed by an image image

regarding the expressed realityregarding the expressed reality• ToneTone — the attitude or message that the — the attitude or message that the creator of creator of

an image wishes to transmitan image wishes to transmit• IntentionIntention — the effect that an image is — the effect that an image is desired to desired to

produce in a viewerproduce in a viewer

Chapter 2 Theoretical Foundations of Multimedia

Page 37: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Images and MeaningImages and Meaning

• Sign Sign — any physical object to which a — any physical object to which a community community ascribes meaningascribes meaning

• Semiotics or Semiology Semiotics or Semiology — study of signs— study of signs• SignifierSignifier — the physical entity that — the physical entity that expresses the signexpresses the sign• Signified Signified — the concept or emotion that is — the concept or emotion that is conveyed by the conveyed by the

signsign• SignificationSignification — the link between the — the link between the expression and the expression and the

conceptconcept

Chapter 2 Theoretical Foundations of Multimedia

Page 38: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Images and MeaningImages and Meaning

Chapter 2 Theoretical Foundations of Multimedia

To refute the assertion that pictures are arbitrary in their To refute the assertion that pictures are arbitrary in their connection to what they purport to represent, Messaris connection to what they purport to represent, Messaris uses the real-life situation of the powerful and powerless uses the real-life situation of the powerful and powerless camera anglescamera angles

Page 39: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Lester’s pragmatic perspectives for Lester’s pragmatic perspectives for analyzing and interpreting imagesanalyzing and interpreting images

• Personal Personal — gut reaction; subjective— gut reaction; subjective• HistoricalHistorical— importance of the image based on — importance of the image based on the medium’s time linethe medium’s time line• TechnicalTechnical — relationship between light, the — relationship between light, the recording medium, and the recording medium, and the

presentationpresentation• Ethical Ethical — producer’s, subject’s, and viewer’s — producer’s, subject’s, and viewer’s moral and ethical moral and ethical

responsibilities to the workresponsibilities to the work• CulturalCultural — analysis of the work’s symbols that — analysis of the work’s symbols that convey meaning in this society convey meaning in this society

at this timeat this time• CriticalCritical — issues transcending a particular — issues transcending a particular image to shape a reasoned image to shape a reasoned

reactionreaction

Chapter 2 Theoretical Foundations of Multimedia

Page 40: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Still ImagesStill Images

Chapter 2 Theoretical Foundations of Multimedia

Concrete Abstract

Page 41: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Messaris’ ten major discrepancies Messaris’ ten major discrepancies between concrete images and realitybetween concrete images and reality

• Cannot reproduce full range of brightnessCannot reproduce full range of brightness• Cannot reproduce full range of colorsCannot reproduce full range of colors• Lack information about changes in brightnessLack information about changes in brightness• Lack information about the color of objectsLack information about the color of objects• Cannot reproduce a stereoscopic effectCannot reproduce a stereoscopic effect• Cannot reproduce motion parallax effectsCannot reproduce motion parallax effects• May not show diminution of size with increasing distance from the viewerMay not show diminution of size with increasing distance from the viewer• May not be constrained to a single view at a timeMay not be constrained to a single view at a time• May contain major distortions of featuresMay contain major distortions of features• May entail omissions of features of the subjectMay entail omissions of features of the subject

Chapter 2 Theoretical Foundations of Multimedia

Page 42: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Messaris’ ten major discrepancies Messaris’ ten major discrepancies between concrete images and realitybetween concrete images and reality

Messaris’ discrepancies do not present a serious problem. Human beings, even Messaris’ discrepancies do not present a serious problem. Human beings, even from a very young age, show a remarkable capacity to recognize and correctly from a very young age, show a remarkable capacity to recognize and correctly identify even the most stylized or incomplete concrete representations, identify even the most stylized or incomplete concrete representations, regardless of the presence or absence of color, depth cues, or a regardless of the presence or absence of color, depth cues, or a preponderance of easily recognizable objects. preponderance of easily recognizable objects.

The significance of this observation for a multimedia developer is that the The significance of this observation for a multimedia developer is that the expense of creating and displaying extremely high-resolution, realistic expense of creating and displaying extremely high-resolution, realistic graphics generally is not necessary. The lesson is not to waste resources on graphics generally is not necessary. The lesson is not to waste resources on glitz but to devote them to substance. glitz but to devote them to substance.

Chapter 2 Theoretical Foundations of Multimedia

Page 43: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Components of FormComponents of Form

• DotDot — a small circle that has been filled in — a small circle that has been filled in• LineLine — a sequence of dots so close together that there appears to be no space between — a sequence of dots so close together that there appears to be no space between

themthem• ShapesShapes

ParallelogramParallelogram — a four-sided figure with opposite sides parallel and equal in length — a four-sided figure with opposite sides parallel and equal in length CircleCircle — the locus of all points equidistant from a given point — the locus of all points equidistant from a given point TriangleTriangle — a three-sided figure — a three-sided figure

– equilateralequilateral– isoscelesisosceles

Chapter 2 Theoretical Foundations of Multimedia

Page 44: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Arrangements of DotsArrangements of Dots

Chapter 2 Theoretical Foundations of Multimedia

Center ofCenter ofattentionattention

TensionTensionthroughthrough

asymmetryasymmetry

TensionTensionthroughthroughdivisiondivision

Fill inFill inthe dotsthe dots

Page 45: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Arrangements of LinesArrangements of Lines

Chapter 2 Theoretical Foundations of Multimedia

OpenOpenspacespace

ConstrictedConstrictedspacespace

VerticalVerticalhalthalt

DynamismDynamismand and

directiondirection

Page 46: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Arrangements of LinesArrangements of Lines(continued)(continued)

Chapter 2 Theoretical Foundations of Multimedia

NervousNervousenergyenergy

StrengthStrengthandand

confidenceconfidence

Grace andGrace andairinessairiness

ParallelParallellineslines

and textand text

Page 47: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Uses of Shapes —Uses of Shapes —ParallelogramsParallelograms

Chapter 2 Theoretical Foundations of Multimedia

UnbalancedUnbalancedappearanceappearance

Balanced use of white spaceBalanced use of white space

Page 48: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Uses of Shapes —Uses of Shapes —CirclesCircles

Chapter 2 Theoretical Foundations of Multimedia

This circle dominates This circle dominates the field it is on.the field it is on.

This circular device is anThis circular device is animmediate attention-grabberimmediate attention-grabber

Page 49: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Uses of Shapes —Uses of Shapes —TrianglesTriangles

Chapter 2 Theoretical Foundations of Multimedia

Pyramid —Pyramid — symbol of symbol of

strength andstrength andstabilitystability

Isosceles triangleIsosceles trianglepointing uppointing up

evokes image ofevokes image ofa church steeplea church steeple

Isosceles triangleIsosceles trianglestronglystronglyindicatesindicatesdirectiondirection

Page 50: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Graphics PrimitivesGraphics Primitives

• PointPoint — a pixel on a computer screen — a pixel on a computer screen• LineLine — two or more pixels connected (adjacent) to each — two or more pixels connected (adjacent) to each

otherother• Polygon Polygon — a closed figure of more than two sides existing — a closed figure of more than two sides existing

in one planein one plane• The computer domain of straight lines and geometrical The computer domain of straight lines and geometrical

figures cannot reproduce perfect renditions of the real figures cannot reproduce perfect renditions of the real worldworld

Chapter 2 Theoretical Foundations of Multimedia

Page 51: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Factors Providing a Sense of DepthFactors Providing a Sense of Depth

• SpaceSpace — the frame in which an image is set — the frame in which an image is set• SizeSize — apparent sizes of objects provide clues — apparent sizes of objects provide clues• Color Color —— warm colored objects seem closer than cool ones, as do more warm colored objects seem closer than cool ones, as do more

highly contrasted objectshighly contrasted objects• LightingLighting — differences in light intensity and shadowing portray depth — differences in light intensity and shadowing portray depth• Textural GradientsTextural Gradients — e.g., ripple effects become closer together in the — e.g., ripple effects become closer together in the

distancedistance• TimeTime — a culture’s perception of time influences the depiction of depth — a culture’s perception of time influences the depiction of depth• PerspectivePerspective — a drawing technique — a drawing technique

Chapter 2 Theoretical Foundations of Multimedia

Page 52: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Three forms of PerspectiveThree forms of Perspective

Chapter 2 Theoretical Foundations of Multimedia

IllusionaryIllusionaryperspectiveperspective

GeometricalGeometricalperspectiveperspective

ConceptualConceptualperspectiveperspective

Page 53: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Motion in ImagesMotion in Images

• Real movement Real movement — animate and inanimate objects in motion — animate and inanimate objects in motion in the real worldin the real world

• Apparent movementApparent movement —still images providing the —still images providing the appearance of motionappearance of motion

• Persistence of vision Persistence of vision — the retention of an image in the — the retention of an image in the brain for a period of timebrain for a period of time

• In MM, the number of images per second and the viewer’s In MM, the number of images per second and the viewer’s persistence of vision de- termine the appearance of persistence of vision de- termine the appearance of realistic motionrealistic motion

Chapter 2 Theoretical Foundations of Multimedia

Page 54: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Motion in ImagesMotion in Images

• Albert Michotte Albert Michotte — Perception of physical causality— Perception of physical causality launching effectlaunching effect entraining effectentraining effect

• Graphic movementGraphic movement —the motion of the eyes in scanning an —the motion of the eyes in scanning an object (often left-to-right, top-to-bottom); culturally basedobject (often left-to-right, top-to-bottom); culturally based

• Implied movement Implied movement — the illusion of motion in a still image, — the illusion of motion in a still image, without any movement of the image, its surroundings, or the without any movement of the image, its surroundings, or the viewerviewer

Chapter 2 Theoretical Foundations of Multimedia

Page 55: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Early WritingEarly Writing

Chapter 2 Theoretical Foundations of Multimedia

PictographsPictographsIdeographsIdeographs

Page 56: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Early WritingEarly Writing

Chapter 2 Theoretical Foundations of Multimedia

HieroglyphicsHieroglyphics LogogramsLogograms

Early AlphabetsEarly Alphabets CuneiformCuneiform

Page 57: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

TypographyTypography

Chapter 2 Theoretical Foundations of Multimedia

Movable Type

A pagefrom aGutenbergBible

Courtesy of specialCourtesy of specialcollections, M. I.collections, M. I.King Library, Uni-King Library, Uni-versity of Kentuckyversity of Kentucky

Page 58: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

TypographyTypography

Chapter 2 Theoretical Foundations of Multimedia

Movable Type

Same pagefrom aGutenbergBible showingcolored inks and illuminations

Courtesy of specialCourtesy of specialcollections, M. I.collections, M. I.King Library, Uni-King Library, Uni-versity of Kentuckyversity of Kentucky

Page 59: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typefaces —Typefaces —Serif TypefacesSerif Typefaces

Chapter 2 Theoretical Foundations of Multimedia

AGaramondAGaramond PalatinoPalatino

Bookman Old StyleBookman Old StyleCourierCourier

Footlight MT LightFootlight MT LightNew YorkNew York

Page 60: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typefaces —Typefaces —Sans Serif TypefacesSans Serif Typefaces

Chapter 2 Theoretical Foundations of Multimedia

ArialArialArial NarrowArial Narrow

Century GothicCentury GothicChicagoChicagoHelveticaHelveticaImpactImpact

Page 61: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typefaces —Typefaces —Special TypefacesSpecial Typefaces

Chapter 2 Theoretical Foundations of Multimedia

BraggadocioBraggadocioBrush Script MTBrush Script MT

DesdemonaDesdemonaMistralMistralStencilStencil

(symbol)(symbol)

Page 62: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typefaces —Typefaces —Type StylesType Styles

Chapter 2 Theoretical Foundations of Multimedia

BoldItalic

OutlineUnderlineShadowShadowSMALL CAPS

Page 63: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typefaces —Typefaces —Weight, Size, and x-heightWeight, Size, and x-height

Chapter 2 Theoretical Foundations of Multimedia

Aa Aa Aa Aa Aa AaDifferent weights and stresses

Aa Aa Aa Aa Aa Aa Aa AaFont sizes from 12 points to 96 points

hd hd hd hd hd hdVarying x-height

Page 64: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typography —Typography —AlignmentAlignment

Chapter 2 Theoretical Foundations of Multimedia

Left Justified Right justified Justified Centered

Page 65: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Typography —Typography —Special Type TechniquesSpecial Type Techniques

Chapter 2 Theoretical Foundations of Multimedia

Page 66: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Production of SoundProduction of Sound

• EquilibriumEquilibrium— An object at rest; the mid-point of the — An object at rest; the mid-point of the motion of vibrationmotion of vibration

• Independence of magnitude and period Independence of magnitude and period —accounts for —accounts for the independence of volume and pitchthe independence of volume and pitch

• Speed of a point in vibration varies Speed of a point in vibration varies — as vibration — as vibration proceeds, a point reaches maxi-mum speed as it passes proceeds, a point reaches maxi-mum speed as it passes the equilibrium point, then slows until it reaches it maxi-the equilibrium point, then slows until it reaches it maxi-mum swing, reverses itself and begins to speed up againmum swing, reverses itself and begins to speed up again

Chapter 2 Theoretical Foundations of Multimedia

Page 67: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Sound WavesSound Waves

• Frequency Frequency — number of vibrations per second; — number of vibrations per second; determines the pitchdetermines the pitch

• AmplitudeAmplitude—the intensity or volume of a heard sound, or —the intensity or volume of a heard sound, or the signal strength of an electronically transmitted soundthe signal strength of an electronically transmitted sound

• Harmonics Harmonics — waves having frequencies related by — waves having frequencies related by arithmetic ratios; account for the different sounds of arithmetic ratios; account for the different sounds of different instru-ments, even though they are playing the different instru-ments, even though they are playing the same notesame note

Chapter 2 Theoretical Foundations of Multimedia

Page 68: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Harmonic MotionHarmonic Motion

Chapter 2 Theoretical Foundations of Multimedia

At restAt rest

CompressingCompressing

RarefyingRarefying

Wave motionWave motion

Page 69: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Fourier Spectrum AnalysisFourier Spectrum Analysis

• Baron Jean Baptiste Joseph FourierBaron Jean Baptiste Joseph Fourier• ““Any vibration that repeats itself indef-initely can be built Any vibration that repeats itself indef-initely can be built

up from (or is analyz-able into) a set of pure sinusoid up from (or is analyz-able into) a set of pure sinusoid waves”waves” Handel (1993)Handel (1993)

• ““Any periodic waveform, no matter how complex, can be Any periodic waveform, no matter how complex, can be analyzed, or decom-posed, into a set of simple sinusoid analyzed, or decom-posed, into a set of simple sinusoid waves with calculated frequencies, amplitudes, and phase waves with calculated frequencies, amplitudes, and phase angles”angles” Handel (1993)Handel (1993)

Chapter 2 Theoretical Foundations of Multimedia

Page 70: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Fourier Spectrum AnalysesFourier Spectrum Analyses

Chapter 2 Theoretical Foundations of Multimedia

Page 71: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Propagation of SoundPropagation of Sound

• ReflectionReflection— sound waves bounce off of surfaces and — sound waves bounce off of surfaces and continue to travel in a new directioncontinue to travel in a new direction

• Direct sound Direct sound —sound heard after no reflections—sound heard after no reflections

• Early sound Early sound — sound heard after only one reflection— sound heard after only one reflection

• ReverberationReverberation — sound heard after several reflections — sound heard after several reflections

Chapter 2 Theoretical Foundations of Multimedia

Page 72: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Propagation of SoundPropagation of Sound

Chapter 2 Theoretical Foundations of Multimedia

Page 73: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Doppler EffectDoppler Effect

• When either the source or the listener isWhen either the source or the listener is in motion in motion relative to the other, the wave-length of the soundwaves is relative to the other, the wave-length of the soundwaves is affected, causing the pitch to seem to varyaffected, causing the pitch to seem to vary

• The pitch becomes increasingly higher as the source The pitch becomes increasingly higher as the source approaches the listenerapproaches the listener

• The pitch becomes increasingly lower as the source The pitch becomes increasingly lower as the source recedes from the listenerrecedes from the listener

• The relative motion must be at least 20 or 30 miles per The relative motion must be at least 20 or 30 miles per hourhour

Chapter 2 Theoretical Foundations of Multimedia

Page 74: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Listening and Auditory Listening and Auditory CommunicationCommunication

• Hearing Hearing — merely receiving sound physi-cally, not — merely receiving sound physi-cally, not consciouslyconsciously

• Listening Listening — “is perceiving sound with careful and — “is perceiving sound with careful and responsive discrimination. It is thinking about sound — responsive discrimination. It is thinking about sound — analyzing its quality, style, interpretation, and nuance. It is analyzing its quality, style, interpretation, and nuance. It is trying to understand what motivates a sound. It is trying to understand what motivates a sound. It is engaging in new sonic exper-iences regardless of their engaging in new sonic exper-iences regardless of their strangeness. It is examining your reaction to sound in strangeness. It is examining your reaction to sound in relation to your mood and feeling.” relation to your mood and feeling.” Alten, 1994Alten, 1994

Chapter 2 Theoretical Foundations of Multimedia

Page 75: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Reception of SoundReception of Sound

• Habituation Habituation — 85% of all primary auditory neurons become — 85% of all primary auditory neurons become less sensitive to sounds if they persist unchanged. less sensitive to sounds if they persist unchanged. Musicians use changes in key and vibrato to counteract Musicians use changes in key and vibrato to counteract this phenomenonthis phenomenon

• Pitch Pitch — Most adults can detect pitches between 20 and — Most adults can detect pitches between 20 and 20,000 Hz, but pitches above 16,000 Hz are perceived as 20,000 Hz, but pitches above 16,000 Hz are perceived as hiss. hiss.

• Relationship to age Relationship to age — As age increases, range of hearing — As age increases, range of hearing decreases.decreases.

Chapter 2 Theoretical Foundations of Multimedia

Page 76: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Chapter 2 Theoretical Foundations of Multimedia

Page 77: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Comparison of Changes in Decibels Comparison of Changes in Decibels and Powerand Power

Chapter 2 Theoretical Foundations of Multimedia

dB dB PowerPowerdB dB PowerPower00 11 15 15 32 3211 1.31.3 18 18 64 6422 1.61.6 20 20 100 10033 22 30 30 1,000 1,00044 2.52.5 40 40 10,000 10,00055 3.23.2 50 50 100,000 100,00066 44 60 60 1,000,000 1,000,000 77 55 70 70 10,000,000 10,000,00088 66 80 80 100,000,000 100,000,00099 88 90 90 1,000,000,000 1,000,000,000

1010 10 10 100100 10,000,000,000 10,000,000,000 1111 12 12 110110 100,000,000,000 100,000,000,000 1212 16 16 120 1,000,000,000,000120 1,000,000,000,000

Page 78: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Perception of SoundPerception of Sound

• Hearing Levels Hearing Levels — physical, abstract, etc.— physical, abstract, etc.• Meaning Meaning — is derived from relationships among the — is derived from relationships among the

various levels on which humans hear soundsvarious levels on which humans hear sounds• Localization Localization — is the ability to locate a sound — is the ability to locate a sound

source in three-dimensional spacesource in three-dimensional space• Identification Identification — is the ability to recognize voices, — is the ability to recognize voices,

instruments, and other sources of soundsinstruments, and other sources of sounds

Chapter 2 Theoretical Foundations of Multimedia

Page 79: Light and Sound, Seeing and Listening 2.1 Perception 2.2 Light 2.3 Color and Color Sensation 2.4 Sight and Visual Communication 2.5 Sound 2.6 Listening

Handel’s Approaches to Handel’s Approaches to Conceptualizing Perception of SoundConceptualizing Perception of Sound

• Prototype or template Prototype or template — compare what is heard with an — compare what is heard with an idealized versionidealized version

• Feature or attribute Feature or attribute — abstract critical features to — abstract critical features to distinguish among possibilitiesdistinguish among possibilities

• Higher-order variable Higher-order variable — time-varying, complex acoustic — time-varying, complex acoustic properties that uniquely identify an eventproperties that uniquely identify an event

• Innate systems Innate systems — distinct brain structures that yield — distinct brain structures that yield invariant preceptsinvariant precepts

Chapter 2 Theoretical Foundations of Multimedia