Introduction Prof. Lina Karam School of Electrical ...lina.faculty.asu.edu/eee508/Lectures/eee508_Intro.pdf · 1 EEE 508 - Digital Image & Video Processing and Compression Introduction

1

EEE 508 - Digital Image & Video Processing and Compression

Introduction

Prof. Lina KaramSchool of Electrical, Computer, & Energy Engineering

Arizona State [email protected]

http://lina.faculty.asu.edu/eee508/

Why Image and Video ?

Sample image-and video-based applications • Entertainment • Communications• Medical imaging• Security• Monitoring• Visual sensing and control

2

Basic Imaging System

Copyright 2007 ‐2012 by Lina J. Karam 3

CAMERA Imaging Device

Imaged Scene

DIGITIZER STORAGE PROCESSDisplay, Analysis, Enhancement, Restoration, Compression for transmission

Sampling + Quantization

Compression

Colored lights from scene are captured into red, green, and blue pixels (picture elements)

Scene viewed through “color” filters that separate the image into 3 color components

Digital camera systems contain optics that image light onto sensors typically a CCD array with filters

x

y

z


Copyright 2007‐2012 by Lina J. Karam 4


Imaged Scene



Compression

Quality of captured image depends on imaging optics and electronics, “color” filter characteristics, digitization, and processing

x

y

z

z




Imaged Scene


Compression

High-end digital cameras make use of dichroic filters to split the light into, red, green, and blue components

• Beam splitter is used to split light into three beams that are directed through filters that filter out all but one color for each chip (“dichroic” indicates that 2 out of the 3 colors are filtered).

• Each color component is imaged separately onto an array of sensors: one chip “sees” red (R), one “sees” green (G), and one “sees” blue (B).

• Three values (R, G, B) captured at each pixel position

x

y

z




Imaged Scene



Compression

Common digital cameras have a single imaging element (typically one CCD chip) and make use of tiled Color Filter Array (CFA)

x

y

z

Light from scene

CFA

Image Sensor

Bayer CFAEach captured pixel is eitherGreen (G), Red (R), or Blue(B).Interpolation is used to recover(R,G,B) values at each pixel.

Digitization: Sampling and Quantization



Imaged Scene


Sampling + Quantization CompressionI(x,y;t)

I(n1,n2;n3)x

y

• Original Imaged Scene : analog (continuous in space and time) I(x,y;t) for video and I(x,y) for still imageI: image intensity and color at position (x,y) and at time t

• Digitized sensed image/video: digital (sampled in space and time, plus discrete amplitudes) I(n1,n2;n3) for video and I(n1,n2) for a still imageI: image intensity and color at integer sample position (n1,n2) and

integer time index n3

t




Imaged Scene



Compression

x

• I(s1,s2,s3,t) : ANALOG SIGNALI : real value or vector of real values(s1,s2 ,s3,t) : set of real continuous space (time) variables

• I(n1,n2, n3,n4) : DISCRETE SIGNAL (DIGITAL)I : discrete (quantized) real or integer value(n1,n2,n3,n4 ) : set of integer indices

z

S3 or z

S1or x

t

S2 or y

9#EEE 508 - Lecture 1 9

• Sampled Black & White Photograph: I(n1,n2)I (n1,n2) scalar indicating pixel intensity at location (n1,n2) For example: I = 0 Black

I = 1 White0 < I < 1 In-between

• Sampled color video/TV signal

IR(n1, n2, n3) IR(n1, n2, n3 , n4) 2D TV: IG(n1, n2, n3) ; 3D TV: IG(n1, n2, n3 , n4)

IB(n1, n2, n3) IB(n1, n2, n3 , n4)

Examples



Video Sampling • Temporal sampling affects frame (image) rate and perceived

motion quality.– 50 to 60 frames per second produce smooth apparent motion– 25 (PAL) or 30 (NTSC) frames per second is standard for

television pictures; interlacing can be used to improve the appearance of motion

• Frame rate can be referred to as temporal resolution.



Video Sampling • Progressive and Interlaced Sampling

– Progressive sampling: all lines (rows) in a frame are sampled

– Interlaced sampling: alternate between sampling the odd rows (odd field) for one frame followed by the sampling the even rows (even field) for next frame

Odd or Top Field Even or Bottom Field

Spatial ResolutionA digital image is represented as a rectangular array of picture elements (pixels or pels).

Spatial resolution commonly refers to the number of pixels in the horizontal and vertical directions.


503 pixels

365pixels

503x365 pixelsTotal pixels = 183,595

Spatial Resolution


Video Formats based on Resolution1920

1080High Definition (HDTV) 2 Mega pixels

1280

720High Definition (HDTV) 1 Mega pixels

720

480480i SDTV, 345 Kilo pixels

352

288

176

144CIF, 101 Kpixels

QCIF

576SDTV, 415 Kilo pixels

Spatial Resolution


4CIF: 704x576

CIF: 352x288

QCIF: 176x144

SCIF: 128x96

Spatial ResolutionChoice of frame resolution depends on application and on available storage and transmission capacity.

Perceived resolution refers to the maximum number of line pairs that can be resolved on the display screen or to the smallest details that can be resolved.

• Depends on viewing distance.• Depends on display

Display resolution is commonly expressed in pixels per inch.


Aspect RatioAspect ratio is the ratio of the image’s width to its height.


SDTV Video4:3

1.33:1

Widescreen SDTVHDTV16:9

1.78:1

720

480

1280

720

Full HDTV

1.78:1

1920

1080


How do we process images?

• Exploit visual perception properties• Use/Develop image/video processing

(computer vision) algorithms• Use DSP concepts as tools


How many possible images are there?

We represent pixels as amplitude values (gray scale). 256 levels

1 0

128 levels

1 0

64 levels

1 0

32 levels

1 0

How much to sample (quantize) the gray scale?Humans can distinguish in the order of 100 levels of gray.


How many possible images are there?An image has pixels and dimensions, say 200x200 and assume 64

pixel values (64 gray levels).• A 1x1 image → about 64 images• A 1x2 image → about (64)2 images• A 200x200 image → about (64)40000 images• A large but finite number due to human perceptive

properties.

Documents

Introduction Prof. Lina Karam School of Electrical ...lina.faculty.asu.edu/eee508/Lectures/eee508_Intro.pdf · 1 EEE 508 - Digital Image & Video Processing and Compression Introduction