1

Multimedia InformationMultimedia InformationRepresentationRepresentation

2

Analog SignalsAnalog Signals

Fourier transform and analysis

Analog signal and frequency components

Signal bandwidth and effect of bandlimiting channel

3

Signal encoder designSignal encoder design

Signal encoder

Associated wave form

Nyquist sampling theorem In order to obtain an accurate representation of a time-varying analog

signal, sampling rate >= 2 x highest sinusoidal frequency component Nyquist rate

4

Bandlimiting FilterBandlimiting Filter

Undersampling (< Nyquist rate) results in additional FC being generated that are not present in the original signal Cause the original signal

to become distorted Need bandlimiting

(antialiasing) filter

5

Quantization IntervalsQuantization IntervalsQuantization Interval

q = 2Vmax / 2n

Dynamic range of signal

D = 20log10(Vmax/Vmin)dB

Source of errors

Noise polarity

6

Decoder DesignDecoder Design

7

TextText

Unformatted text: plain text ASCII character set Mosaic character set used in Videotex and Teletext

Formatted text: richtext Hypertext

Pages and hyperlinks HTML, SGML

8

Images: GraphicsImages: Graphics

Graphics: computer-generated images Representation of a computer graphics

High-level version: e.g) PostScript Bit-map format: GIF, TIFF

9

Images: Digitized DocumentsImages: Digitized Documents

10

Images: Digitized PicturesImages: Digitized Pictures

Color principles

Additive color mixing

Subtractive color mixing

11

Raster-scan principlesRaster-scan principles

12

Raster-scan Display ArchitectureRaster-scan Display Architecture

Pixel depth: # of bits per pixel CLUT (color look-up table)

13

Screen ResolutionScreen ResolutionVisible lines per frame

Digitization spatial resolution

14

Digital Cameras and ScannersDigital Cameras and ScannersColor image capture

RGB signal generation alternatives

15

AudioAudio

Speech signals: 50Hz ~ 10kHz Nyquist rate = 20ksps

Music-quality audio: 15Hz ~ 20kHz Nyquist rate = 40ksps

16

PCM SpeechPCM SpeechSignal encoding and decoding

Compressor Expander

U-law : North America, JapanA-law: Europe and others

Obtain 12-bit performance!!

17

CD-quality AudioCD-quality Audio CD-DA (Digital Audio)

standard Sampling rate: 44.1 ksps 16 bits/sample Bit rate for stereo music

2 channel x 44.1 ksps x 16 bits = 1.411 Mbps

18

Synthesized audioSynthesized audio

MIDI (Music Instruments Digital Interface) A MIDI message = (status

byte, data bytes, … ) Music: sequece of

message Play out by the sequencer

program or saved in a file Need sound card

19

Video ApplicationsVideo Applications

Entertainment Broadcast TV, VCR/DVD recording

Interpersonal Video telephony, videoconferencing

Interactive Windows containing short video clips

20

Interlaced ScanningInterlaced Scanning

21

TV broadcast: Color SignalsTV broadcast: Color Signals

Requirements Color TV broadcast should be used by an existing

monochrome (back-and-white) TV set Color TV should be able to produce black-and-white

pictures from monochrome broadcast

Properties of a color source Brightness: amount of energy that stimulates the eye

Varies on a grayscale from back through to white (highest) Independent of the color of the source

Hue( 색조 ): actual color of the source Each color has a different frequency/wavelength

Saturation( 채도 ): strength or vividness of the color

22

Luminance and ChrominanceLuminance and Chrominance

Luminance: related to the brightness Amount of white light Same as the signal used by monochrome TV Ys = 0.299Rs + 0.587Gs + 0.114Bs

Chrominance: related to the hue and saturation Color difference signals i.e no brightness info.

Cb = Bs – Ys,

Cr = Rs – Ys

23

Composite video signalComposite video signal

Magnitude of the two color differences are both scaled down If the two color difference signals are transmitted, amplitude of

luminance signal can become greater than that of the equivalent monochrome signal

PAL Y = 0.299R + 0.587G + 0.114B U = 0.493(B-Y) V = 0.877(R-Y)

NTSC Y = 0.299R + 0.587G + 0.114B I = 0.74(R-Y) – 0.27(B-Y) Q = 0.48 (R-Y) + 0.41(B-Y)

24

Signal BandwidthSignal Bandwidth

NTSC

PAL

25

TV BroadcastTV Broadcast

26

TVTV Analog TV

Frame rate: # of frames per second produced by a video signal common rate: 25 - 75Hz

Number of scan lines per frame scan rate: # of lines scanned per second

= frame rate x # of scan lines Interlacing: to reduce the flickering

odd field, even field

Aspect ratio: width : height

Digital TV convert analog signal to digital signal sampling, quantization, coding

27

Standard Analog TV SystemsStandard Analog TV Systems

Standard Frame rates Scan lines Aspect ratio Interlacing

NTSC 29.97 525(480) 4:3 2:1

PAL 25 625(576) 4:3 2:1

SECAM 25 625(576) 4:3 2:1

28

Digital Video (I)Digital Video (I) 4:2:2 format

Studio-Quality TV: ITU-R 601 Bandwidth 6MHz

sampling rate: 13.5MHz 720 samples / line

Bit rate = 216Mbps (PAL, NTSC)

Subsampling 4:2:2

29

Digital Video (II)Digital Video (II) 4:2:0 format

Bit rate = 162Mpbs

HDTV format 16/9 aspect ratio: 1920 x 1152 pixel Refresh rate: 25Hz / 30Hz Studio application: 4:2:2 format Broadcast application: 4:2:0 format

30

Digital Video (III)Digital Video (III) SIF (source intermediate format)

VCR quality Used in MPEG-1 ½ resolution in both horizontal

and vertical directions (by subsampling): 4:1:1 format

½ refresh rate (temporal resolution)

Bit rate: 81Mbps CIF (common intermediate

format) Videoconferencing quality Used in H.261 4:1:1 format Y = 360 x 288, Cb = Cr = 180 x

144 30Hz

QCIF (quarter CIF) Video Phone Quality Used in H.261

31

PC VideoPC Video

Digitization

FormatSystem

Spatial resolutionTemporal resolutionY Cb = Cr

4:2:0 525-line 640 x 480 320 x 240 60Hz625-line 768 x 576 384 x 288 50Hz

SIF 525-line 320 x 240 160 x 240 30Hz625-line 384 x 288 192 x 144 25Hz

CIF 384 x 288 192 x 144 30Hz

QCIF 192 x 144 96 x 72 15/7.5 Hz