View
227
Download
0
Embed Size (px)
Citation preview
2
• Conversion of energy into vibrations in the air.
• Most sound sources vibrate in complex ways leading to sounds with components at several different frequencies.
• Frequency spectrum – relative amplitudes of the frequency components.
• Range of human hearing: roughly 20Hz–20KHz, falling off with age.
Sound
T.Sharon-A.Frank
3
Use of Audio
1. Convey Information2. Set a mood3. Capture attention4. Explain a process5. Provide personal contact6. Tell a story
T.Sharon-A.Frank
4
Nature of Sound
• Sound is produced by the vibration of matter.
• Pressure variations are created in air surrounding matter.
• Pattern of oscillation is called a waveform.
5
• Sounds change over time– e.g. musical note has attack and decay, speech
changes constantly.
• Frequency spectrum alters as sound changes.
• Waveform is a plot of amplitude against time:– Provides a graphical view of characteristics of a
changing sound.– Can identify syllables of speech, rhythm of music,
quiet and loud passages, etc.
Waveforms
T.Sharon-A.Frank
6
Basic Sound Concepts
• Waveform:– Period
– Frequency
– Amplitude
• Computer Representation of Sound:– Sampling
– Quantization
T.Sharon-A.Frank
7
Period
• Repeating portion of the sound wave, although sound waves are never perfectly smooth or uniformly periodic.
T.Sharon-A.Frank
8
Frequency
• Reciprocal of the period, that is, number of periods in a second; measured in hertz (Hz) or cycles per second.
• Human hearing frequency range is from 20Hz to 20KHz:
– Infrasound: 0 - 20Hz
– Ultrasound: 20KHz - 1GHz
– Hypersound: 1GHz - 10THz
T.Sharon-A.Frank
9
Amplitude
• Measure of displacement of the air pressure wave from its mean.
• Subjectively heard as loudness.
amplitude
T.Sharon-A.Frank
11
0
0.2
0.4
0.6
0.8
1
1.2
1 4 7 10 13 16 19 22 25 28 31 34 37
Series1
0
0.2
0.4
0.6
0.8
1
1.2
1 4 7 10 13 16 19 22 25 28 31 34 37
Series1
Idea of Audio Digitization
• Smooth continuous curve of waveform is not directly represented in computer.
• Amplitude of waveform is measured and recorded at regular time intervals - process is called encoding.
• Each interval is called a sample.
• Need Analog-to-Digital Converter (ADC).
12
• Converting signal from analogue to digital form
– Analogue signal can vary continuously; digital is restricted to discrete values.
• Two-stage process:
– Sampling – measure the value at discrete intervals.
– Quantization – restrict the value to a fixed set
of quantization levels.
Audio Digitization
T.Sharon-A.Frank
15
Audio Encoding
(a) A sine wave. (b) Sampling the sine wave. (c) Quantizing the samples to 4 bits.
T.Sharon-A.Frank
17
Sampling Rate
• Number of samples per second measured in hertz (Hz).
• More samples = higher quality = bigger files.
• Example:
– CD quality = 44,100 Hz (44.1 KHz)
– Others: 8,000 Hz, 11,025 Hz, 22,050 Hz
T.Sharon-A.Frank
18
Sampling Size
• Number of bits used to represent each sample.
• Called also Bit Depth and Sound Resolution.
• 8-bit - telephone quality (typical of Web).
• 16-bit - CD quality.
• Examples:
16 bit, 44.1kHz
8 bit , 44.1KHz
Web quality (11.025 KHz, 8bit)
T.Sharon-A.Frank
19
Channels
• Audio files can support from 1 to 6 channels of audio information.
• Most common:– mono (1 channel)
– stereo (2 channels)
• 4-6 channels are usually used for surround sound.
T.Sharon-A.Frank
21
Sample Audio File Formats
• aif/aiff
• wav
• au
• mov
• MPEG/MP3 (MPEG 1 Level 3)
• RA/RAM• MIDI (Musical Instrument Digital Interface)
T.Sharon-A.Frank
22
Audio File Formats (platform specific)
• aif (aiff)– Audio Interchange File Format– original audio format for Mac– supported by Windows and other platforms
• wav– Waveform Audio File Format– original Windows audio file format– supported on Mac
• au – u-law (myoo-law) – Unix standard audio format
T.Sharon-A.Frank
23
Audio File Formats (container formats)
• mov (qt)– QuickTime, not just Apple/MAC
• mpg (mpe, mpeg)– Moving Pictures Expert Group– High quality, excellent compression file format– Files tend to be small and sound good
• ra (ram)– RealAudio
T.Sharon-A.Frank
24
Real Audio Format
• First example of streaming audio on the Web.
• Quality like that of an AM radio station.
• Best used for narration and not for music or other sounds.
• Must have RealPlayer on your system to listen to files.
25
• MIDI = Musical Instruments Digital Interface
• Instructions about how to produce music, which can be interpreted by suitable hardware and/or software.
• Standard protocol for communicating between electronic instruments (synthesizers, samplers, drum machines).
• Allows instruments to be controlled by hardware or software sequencers.
MIDI
26
• MIDI interface allows a computer to send MIDI data to instruments.
• Store MIDI sequences in files, exchange them between computers, incorporate them into multimedia.
• Computer can synthesize sounds on a sound card, or play back samples from disk in response to MIDI instructions– Computer becomes primitive musical instrument
(quality of sound inferior to dedicated instruments).
MIDI and Computers
T.Sharon-A.Frank
27
Uncompressed Audio Digital Telephony
• Sampled at 8KHz (8K cycles per second)
• Quantized to 8 bits per sample (1 Byte)
=> 64,000bits/s
• Bandwidth = 64,000/1,024 =
Total: 62.5Kb/s
T.Sharon-A.Frank
28
Uncompressed Stereo Audio CD Quality
• Sampled at 44.1KHz (44.1K cycles per second)
• Quantized to 16 bits per sample (2 bytes)
• Two channels for stereo
• Bandwidth =
2 * 44,100 * 16 = 1,411,200b/s
1,411,200/1024= 1,378Kb/s
Total: ~1.4Mb/s
29
• The creation of moving pictures one frame at a time:
– Literally 'to bring to life‘.
– e.g. make a sequence of drawings on paper, in which a character's position changes slightly in each drawing.
– Photograph drawings in sequence, using movie camera that advances one frame at a time.
– Play back resulting film – character will move.
Animation
T.Sharon-A.Frank
30
Use of Animation
1. Capture the viewer’s attention, entertain, or promote.
2. Explain a system or process.
3. Set a mood or create an on-screen environment.
T.Sharon-A.Frank
31
Animation Principles
• Persistence of vision:– object seen by human eye remains mapped on
retina for a brief time after viewing.– display series of images rapidly and they blend
together to create illusion of movement.– requires at least 16 frames per second (fps) to look
seamless.
• Television: displayed at 30 frames per second.
• Movies: displayed at 48 frames per second.
T.Sharon-A.Frank
32
Cell Animation (Frame Animation)
• Start with first and last image in motion (called keyframes).
• Draw images between keyframes (process called tweening/morphing) - small changes between images.
• Images are then layered onto background scene.
• Images displayed rapidly
– 15/24/30 frames per second.
33
• Logic and procedural concepts are same as in cell animation (keyframe, tweening, layering techniques).
• But use computer and video camera.
• Frame grabbing – record each frame to disk.
• Edit non-linearly like for video.
• Can also use scanner or digital still camera, or create each frame in a graphics program (e.g., Painter).
Captured Animation
T.Sharon-A.Frank
34
Cell-based Techniques
• Morphing:– uses frames to create illusion of one object
changing into another– more keyframes = smoother morph.
• Need at least 15 frames per second.• Professional quality uses 24-30 frames per second.• Cell-based animation is done more easily on computer
than by hand.• Easier to create and process than movies.
T.Sharon-A.Frank
35
Path-based Animation (Vector Animation)
• Create animated objects by following object’s transition over a line or vector.
• Artist creates one drawing and a path.
• Computer program manipulates object by drawing frames as object travels over the path.
T.Sharon-A.Frank
36
Sample Animation File Formats
• Animated GIF
• mov (QuickTime)
• SWF (Flash)
• PNG (PortableNetwork Graphics)
37
• Sequence of images can be stored in a single GIF file, and displayed one after another by a Web browser or other software:
– No browser plug-in required
– Can specify looping, delay between frames
– 256 colour palette
– No sound.
Animated GIFs
38
• Popular Web animation format.
• Usually generated by Macromedia Flash.
• Vector animation format
– Motion represented as numerical operations on vector data.
• Can also include bitmapped images (e.g., as backgrounds).
SWF
T.Sharon-A.Frank
39
Video
• Video is sequence of images displayed at constant rate.
• Rate is in frames per second (fps).
• Needs around 10-15 fps for smooth motion.
• For example, video is displayed at 24-30 fps.
T.Sharon-A.Frank
40
Use of Video
1. Tell a story
2. Entertain an audience
3. Explain a complex process
4. Personalize a Multimedia experience
T.Sharon-A.Frank
41
Movie - Characteristics
• Built from Frames - Each frame is an Image– Image characteristics (i.e., resolution, colors)
Clip
Scene
ShotFrame Frame
T.Sharon-A.Frank
44
MPEG
• MPEG = Moving Picture Experts Group.• Family of compression algorithms.• High compression rate.• Algorithm stores only changes from one frame
to another rather than each entire frame.• Video info then encoded using DCT.• Lossy compression but loss of data is generally
imperceptible to human eye.
T.Sharon-A.Frank
45
MPEG Video Format
• ISO/IEC MPEG (Moving Picture Experts Group) Series of standards including:
1. MPEG-1: used for video CD 352x240 @30 frames/second.
2. MPEG-2: used for compressing DVD and high-definition television (HDTV).
3. MPEG-4: used to transmit audio, video, and graphics at low bitrate – can be delivered over very slow connections (56 Kbps).
T.Sharon-A.Frank
46
Video Acquisition
• TV set and PC monitor look similar.• TV is analog
– Gets info from continuously varying broadcasting waves.
• PC is digital– Data displayed is digital.
• Digitization performed by video capture card.• Typical card is combination of hardware and software.• Capture card typically creates uncompressed files that
then must be passed into a codec chip or software.
T.Sharon-A.Frank
47
Issues Affecting Video File Size
• Movie Length - simple principle: shorter video clips result in smaller files.
• Frame Size - full-screen video is 640x480. Typical frame size on WWW = 160x120 although can handle 320x240.
• Frame Rate:– TV quality = 30 fps – Web = 15 (or even 10) fps.
49
• NTSC (National Television Standard Committee)
– Used by TV and video industries in US and Japan Taiwan, parts of South America, …
• PAL (Phase Alternating Line)– Used in Western Europe, Australia & New Zealand,
China , …
• SECAM (Sequential Color and Memory)– France and former Soviet Union– Standard only used for transmission– Very different technology than PAL.
TV Standards
T.Sharon-A.Frank
50
Video (PAL, HDTV) Bandwidth
• PAL Resolution: 640 x 480
• 3 bytes per pixel
• Frame rate: 25 fps
• Bandwidth:
640 * 480 * 25 * 24 = 184,320,000b/s
Total: ~176Mb/s
HDTV bandwidth increased by factor of 5.33: ~936Mb/s