Multimedia Specification Design and Production 2012 / Semester 1 / L2 Lecturer: Dr. Nikos Gazepidis [email protected]

Multimedia Specification Design and Production

2012 / Semester 1 / L2Lecturer: Dr. Nikos [email protected]

2

Digital Media

In computers, audio, image and video are stored as files just like other text files.

For images, these files can have an extension like: BMP, JPG, GIF, TIFF, PNG, etc.

For audios, the file extensions include: WAV, MP3, …

The videos files usually have extensions: AVI, MOV, MKV

Digital Audio, Image & Video

3

An Example of Digital Image

Let’s open an image file is its “raw” format:


* P6: (this is a ppm image)* Resolution: 512x512* Depth: 255 (8bits per pixel in each channel)

4

An Example of Digital Image (cont.)


Color depth or Bit depth is the number of bits used to indicate the color of a single pixel in a bitmapped image or video frame buffer.

1-bit color (21 = 2 colors): monochrome 2-bit color (22 = 4 colors): CGA, gray-scale 3-bit color (23 = 8 colors): many early home computers with TV displays 4-bit color (24 = 16 colors): least common denominator VGA 5-bit color (25 = 32 colors) 6-bit color (26 = 64 colors) 8-bit color (28 = 256 colors): most early color Unix workstations, VGA at low-res 12-bit color (212 = 4096 colors): some Silicon Graphics systems

The HDMI 1.3 specification defines bit depths of 30 bits (1.073 billion colors), 36 bits (68.71 billion colors), and 48 bits (281.5 trillion colors) …& transparency

5



1 bit (2 colors) 2 bits (4 colors) 4 bits (16 colors)

8 bits (256 colors) 24 bits (16,777,216 colors, "truecolor")

6



8 bits

16 bits

24 bits

7



An image containsa header anda bunch of (integer) numbers.

8

Digital Media Capturing


To get a digital image, an audio or a video clip, we need some media capturing device such as:

a digital camera or a scanner, a digital audio recorder, or a digital camcorder.

All these devices have to complete tasks:

Sampling: To convert a continuous media into discrete formats. Digitization: To convert continuous samples into finite number of digital numbers. There are probably some further compression process.

9

An Audio Signal


10

Sampling for an Audio Signal


0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

Am

plit

ud

e

Sampling period Ts,fs =1/Ts

Signal Period T, f = 1/T

The sampling frequency or sampling rate fs is defined as:

the number of samples obtained in one second, or fs = 1/T

…and is measured in Hertz

11



fs = 2.5f

fs = 1.67f

0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

Am

plit

ud

e

Original signal

A new component is added

This is denotedas aliasing.

0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

Am

plitu

de

12



0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

Am

plit

ude

0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

Am

plitu

de

fs = 2f

There are infinite numberof possible sinwaves going throughthe sampling points

13

Frequency Decomposition


We can use “Fourier Transform” to compute these frequency components.

Nyquist Theorem

14

Image Sampling


The sampling theorem applies to 2D signal (images) too Nearest point interpolation.

15

Image Sampling (original image)


16

Image Sampling (Aliasing due to sampling)


17

Digitization


The samples are continuous and have infinite number of possible values.

The digitization process approximates these values with a fixed number of numbers.

To represent N numbers, we need log2N bits.

18

Digital Audio


You often hear that an audio is 16bits at 44kHz.

44KHz is the sampling frequency. Music has more high frequency components than speech. 8kHz sampling is good enough for telephone quality speech.

16bits means each sample is represented as a 16bit integer.

Digital audio could have more than one channels.

19

Digital Images


An image contains 2D samples of a surface, which can be represented as matrices. Each sample in an image is called a pixel.

20

Types of Digital Images


Grayscale image

Usually we use 256 levels for each pixel. Thus we need 8bits to represent each pixel (2^8 == 256)

Some images use more bits per pixel, for example MRI (Magnetic resonance imaging) images could use 16bits per pixel.

A 8bit grayscale Image.

21

Binary Image


A binary image has only two values (0 or 1).

Binary image is quite important in image analysis and objectdetection applications.

22

Binary Image


[ b7 b6 b5 b4 b3 b2 b1 b0]

Each bit plane is a binary image.

Most Significance Bitplane - MSB

Less Significance Bitplane - LSB

b7

23

Dithering


A technique to represent a grayscale image with a binary one.

Convert image to4 levels: I’ = floor(I/64)

0 1

2 3

24

Dithering Matrix


0 1

2 3

0 23 1The dithering matrix is:

Definition

A square matrix of threshold values that is repeated as a regular array to provide a threshold pattern for an entire image in the dither method of image representation

25

Color Image


r

g

b

There are other color spaces: YUV, HSV etc.

24 bit image

26

Color Table


Image with 256 colors

r

g

b

Clusters of colors

It is possible touse much less colorsTo represent a color imagewithout much degradation.

27

Human Vision


Human eye has two kinds of light sensitive cells.1.The rods and2.The cones Rods response curve

(black and white vision)

Cones response curve(color vision)

R = s E() Sr()dG = s E() Sg()dB = s E() Sb()d

28

Colors


Colorimeter experiment

Color matching function

29

CIE Color Matching Functions


The amounts of R, G, B lighting sources to form single wavelength light forms the color matching curves.

CIE color matching curves CIE standard color matching functions

CIE = International Commission on Illumination

30

Gamma Correction


Linearly increasing intensityWithout gamma correction

Linearly increasing intensitywith gamma correction

31

Video (Analog Video)


Even frameOdd Frame

Definition

Analog video is a video signal transferred by an analog signal.

An analog color video signal contains luminance, brightness (Y) and chrominance (C) of an analog television image.

When combined in to one channel, it is called composite video as is the case, among others with NTSC, PAL and SECAM.

Analog video may be carried in separate channels, as in two channel S-Video (YC) and multi-channel component video formats.

32

Color System in Video


YUV was used in PAL (an analog video standard) and also used for digital video.

Y is the luminance component (brightness) Y = 0.299 R + 0.587 G + 0.144 B

U and V are color components U = B – Y V = R - Y

33

PAL vs NTSC vs SECAM


PAL (Phase Alternating Line), is an analogue television colour encoding system used in broadcast television systems in many countries.

NTSC (National Television System Committee) receivers have a tint control to perform colour correction manually. If this is not adjusted correctly, the colours may be faulty. The PAL standard automatically cancels hue errors by phase reversal, so a tint control is unnecessary.

SECAM (Sequential Color with Memory) is an earlier attempt at compatible colour television which also tries to resolve the NTSC hue problem. It does so by applying a different method to colour transmission, namely alternate transmission of the U and V vectors and frequency modulation, while PAL attempts to improve on the NTSC method. SECAM transmissions are more robust over longer distances than NTSC or PAL.

34

PAL vs NTSC vs SECAM


Documents

Multimedia Specification Design and Production 2012 / Semester 1 / L2 Lecturer: Dr. Nikos Gazepidis [email protected]