Digital Image Processing - University of Houstonqil.uh.edu/dip/media/cosc6380/Lecture_-_4.pdfQUANTIZATION • Each pixel gray level is quantized: assigned one of a finite set of numbers

Digital Image ProcessingCOSC 6380/4393

Lecture – 4

Jan. 23rd, 2020

Slides from Dr. Shishir K Shah and Frank (Qingzhong) Liu

Review: Pre-Introduction

• Example: Measure depth of the water in meters at a certain pier

• Yet another representation

• Image as a mode/format to convey information usually for human consumption

2

Review: WHAT ARE DIGITAL IMAGES?• Images are as variable as the types of radiation that exist and

the ways in which radiation interacts with matter:

3

Image formation

• Let’s design a method to capture reflection

– Idea 1: put a piece of film in front of an object

– Do we get a reasonable image?

4

Light Source

Review: Image formation

• Let’s design a method to capture reflection

– Idea 1: put a piece of film in front of an object

– Do we get a reasonable image?

5

Review: Pinhole camera

• Add a barrier to block off most of the rays

– This reduces blurring

– The opening is known as the aperture

– How does this transform the image?

6

Review: Adding a lens

• A lens focuses light onto the film

– There is a specific distance at which objects are “in focus”• other points project to a “circle of confusion”in the image

– Changing the shape of the lens changes this distance

7

Review: OPTICS OF THE EYE

8

Review: PHOTORECEPTORS

• Rods are 1-2 microns in diameter; the cones are 2-3 microns in diameter in the fovea, but increase in diameter away from the fovea (No rods in the fovea)

• Cones are densely packed in the fovea and quickly decrease in density as a function of eccentricity

• Rods increase in density out to approximately 20 degree eccentricity, beyond which their density begins to decline

9

Review: OPTICAL IMAGING GEOMETRY

• We will quantify how the geometry of a 3-D scene projects to the geometry of the image intensities:

object

lens

image

sensingplate,

emulsion, etc

light source(point source)

emitted rays

reflectedrays

focallength

10

UPRIGHT PROJECTION GEOMETRY

X

Y

Z

lens center

f = focal lengthimage plane

Upright Projection Model

x

y

(X, Y, Z) = (0, 0, 0)

11

SOLVING PERSPECTIVE PROJECTION• Using similar triangles we can solve for the relationship

between 3-D coordinates in space and 2-D image coordinates• Redraw the imaging geometry once more, this time making

apparent two pairs of similar triangles:

b

f

B

C

a

f

A

C

a

b

A

B

C

f

12

IMAGE ACQUISITION

object

lens

image

sensingplate,

emulsion, etc


emitted rays

reflectedrays

focallength

13

14

How Do We Generate A Digital Image?

• Start with a picture of something

Slide by K. R. Castleman

15



• Lay a grid over the picture


16




• Measure the brightness/intensity in each of the squares


A Simple Image Formation Model

object

lens

image

sensingplate,

emulsion, etc


emitted rays

reflectedrays

focallength

17

i(x, y)

f(x, y)

r(x, y)

Weeks 1 & 2 18

A Simple Image Formation Model

( , ) ( , ) ( , )

( , ) : intensity at the point ( , )

( , ) : illumination at the point ( , )

(the amount of source illumination incident on the scene)

( , ) : reflectance/transmissivity

f x y i x y r x y

f x y x y

i x y x y

r x y

at the point ( , )

(the amount of illumination reflected/transmitted by the object)

where 0 < ( , ) < and 0 < ( , ) < 1

x y

i x y r x y

𝑓 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 𝑟(𝑥, 𝑦)

Weeks 1 & 2 19

Some Typical Ranges of Reflectance

• Illumination - i(x, y)Lumen — A unit of light flow or luminous flux Lumen per square meter (lm/m2) — The metric unit of measure for

illuminance of a surface– 90,000 lm/m^2 clear day– 10,000 lm/m^2 cloudy day– 1,000 lm/m^2 Indoor Office– 0.1 lm/m^2 clear evening

• Reflectance - r(x, y)

– 0.01 for black velvet– 0.65 for stainless steel– 0.80 for flat-white wall paint – 0.90 for silver-plated metal– 0.93 for snow

Representation of intensity

• If 𝑙 = 𝑓 𝑥, 𝑦

• Let 𝐿𝑚𝑖𝑛 ≤ 𝑙 ≤ 𝐿𝑚𝑎𝑥

• Using previous intensities,

– We may expect, 𝐿𝑚𝑖𝑛 ≅ 10 & 𝐿𝑚𝑎𝑥 ≅ 1000, for Indoor

• 𝐿𝑚𝑖𝑛, 𝐿𝑚𝑎𝑥 → 𝑔𝑟𝑒𝑦 𝑠𝑐𝑎𝑙𝑒

21




• Measure the brightness in each of the squares


1 2 3 4 5

22




• Measure the brightness in each of the squares

• The resulting array of numbers(digits) is the digital image


23


• Each number represents the brightness (0 – Max) at the corresponding position in the image

• Each number is the “gray level” or “pixel value” of the corresponding pixel.


25

What are the Pixels?

• Every pixel has a location in the image.

• A pixel’s location is specified by it’s row number and column number (x,y address).

• Every pixel has a gray level value.


26

What Happened To Our Tiger?

• We only used 169 pixels (not enough).

• Increase to 26 X 26 pixels

• Here he is with 676 pixels.


27



• 52 X 52

• Here he is with 2704 pixels.


28



• 130 X 130

• Here he is with 16,900.


29



• 260 X 260

• Here he is with 67,600 pixels.


Weeks 1 & 2 30

Image Acquisition

Transform illumination energy into

digital images

Weeks 1 & 2 31

Image Acquisition Using a Single Sensor

Weeks 1 & 2 32

Image Acquisition Using Sensor Strips

Weeks 1 & 2 33

Image Acquisition Process

Weeks 1 & 2 34

Sensor Response Waveform

Transform illumination energy into

digital images

Weeks 1 & 2 36

Response from a raster scan

A / D CONVERSION

• For computer processing, the analog image must

undergo ANALOG / DIGITAL (A/D) CONVERSION -

Consists of sampling and quantization

A / D CONVERSION

• For computer processing, the analog image must

undergo ANALOG / DIGITAL (A/D) CONVERSION -

Consists of sampling and quantization

Sampling• Each video raster is converted from a continuous

voltage waveform into a sequence of voltage samples:

A / D CONVERSION (contd.)

• Video digitizer board interfaces with the video camera

• Some new “all-digital cameras” include A/D inside the camera

Sampled Image• A sampled image is an array of numbers representing the sampled

(row, column) image intensities

• Each of these picture elements is called a pixel

A / D CONVERSION (contd.)

• Typically the image array is square (N x N) with dimensions

that are a power of 2: N = 2 M (for simple computer

addressing)

M = 7 128 x 128 (2 14 ~ 16,000 pixels)

M = 8 256 x 256 (2 16 ~ 65,500 pixels)

M = 9 512 x 512 (2 18 ~ 262,000 pixels)

M = 10 1024 x 1024 (2 20 ~ 1,000,000 pixels)

• Important that the image be sampled sufficiently densely

• Otherwise the image quality will be severely degraded

• This can be expressed mathematically (The Sampling

Theorem) but the effects are very visually obvious

Sampling: Example

VS

169 Samples 67,600 Samples

Review: Representation of intensity

• If 𝑙 = 𝑓 𝑥, 𝑦

• Let 𝐿𝑚𝑖𝑛 ≤ 𝑙 ≤ 𝐿𝑚𝑎𝑥

• Using previous intensities,

– We may expect, 𝐿𝑚𝑖𝑛 ≅ 10 & 𝐿𝑚𝑎𝑥 ≅ 1000

• 𝐿𝑚𝑖𝑛, 𝐿𝑚𝑎𝑥 → 𝑔𝑟𝑒𝑦 𝑠𝑐𝑎𝑙𝑒

QUANTIZATION

• Each pixel gray level is quantized: assigned one of a finite set of

numbers (generally integers indexed from 0 to K-1

• Typically there K = 2 B possible gray levels:

• Each pixel is represented by B bits, where usually 1 B 8

• The pixel intensities or gray levels must be quantized sufficiently

densely so that excessive information is not lost

• This is hard to express mathematically, but again, quantization

effects are visually obvious

DIGITAL IMAGE

REPRESENTATION

• Once an image is digitized (A/D) and stored it is an array of voltage or magnetic potentials

• Not easy to work with from an algorithmic point of view

• The representation that is easiest to work with from an algorithmic perspective is that of a matrix of integers

Matrix Image Representation• Denote a (square) image matrix I = [I(i, j); 0 < i, j < N-1]

where

• (i, j) = (row, column)

• I(i, j) = image value at coordinate or pixel (i, j)

DIGITAL IMAGE

REPRESENTATION (contd.)

• Example - Matrix notation

• Example - Pixel notation - an N x N image

What’s the minimum number

of bits/pixel allocated?

DIGITAL IMAGE

REPRESENTATION (contd.)• Example - Binary Image

(2-valued, usually

BLACK and WHITE)

• Another way of depicting the

image:

Weeks 1 & 2 48

Representing Digital Images

• Discrete intensity interval [0, L-1], L=2k

• Aka. Dynamic Range

• The number b of bits required to store a M × N digitized image

Weeks 1 & 2 49


• Discrete intensity interval [0, L-1], L=2k

• The number b of bits required to store a M × N digitized image

total bits = M × N × k

Weeks 1 & 2 50


Weeks 1 & 2 51

Spatial and Intensity Resolution

• Spatial resolution— A measure of the smallest discernible detail in an image

— stated with line pairs per unit distance, dots (pixels) per unit distance, dots per inch (dpi)

• Intensity resolution— The smallest discernible change in intensity level

— stated with 8 bits, 12 bits, 16 bits, etc.

Weeks 1 & 2 52

Spatial Resolution

Weeks 1 & 2 53

Intensity Resolution

Weeks 1 & 2 54

Spatial and Intensity Resolution

Resampling

• Once the image is acquired.

• How to

– Enlarge an image

– Shrink an image

– Zoom in

• Zooming Example:

– Initial image size = 500 X 500

– Required image size (= X 1.5) = 750 X 750

Zooming

8 X 5 Image

3 X 3 Image

Create a Grid5 X 8 Image

3 X 3 Image

Fill in values by preserving some sense spatial relationship between intensity values

Shrink5 X 8 Image

3 X 3 Image

Overlap5 X 8 Image

3 X 3 Image

Fill in values preserving spatial relationship

How to fill values5 X 8 Image

3 X 3 Image

Fill in values by preserving some sense spatial relationship between intensity values

Weeks 1 & 2 62

Image Interpolation

• Interpolation — Process of using known data to estimate unknown values

e.g., zooming, shrinking, rotating, and geometric correction

• Interpolation (sometimes called resampling) — an imaging method to increase (or decrease) the number of pixels in a digital image.

Some digital cameras use interpolation to produce a larger image than the

sensor captured or to create digital zoom

http://www.dpreview.com/learn/?/key=interpolation

Interpolation: Nearest Neighbor5 X 8 Image

3 X 3 Image


Interpolation: Nearest Neighbor5 X 8 Image

3 X 3 Image


66

Interpolation: Nearest neighbor

Original

Zoom

67

Interpolation: Nearest neighbor

Zoom

Original

Interpolation (1D)

• Known points 𝑥1𝑎𝑛𝑑 𝑥2 with values

• 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑓 → ℝ

• 𝑓 𝑥1 = 𝐼1and 𝑓 𝑥2 = 𝐼2• How to find the value 𝐼 at point 𝑥

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼

Linear Interpolation

• Underlying assumption: 𝑓is linear

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


• Underlying assumption: 𝑓is linear𝑓 𝑧 = 𝑎𝑧 + 𝑏

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼



𝑓 𝑥1 = 𝑎𝑥1 + 𝑏

𝑓 𝑥2 = 𝑎𝑥2 + 𝑏

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼



𝑓 𝑥1 = 𝑎𝑥1 + 𝑏

𝑓 𝑥2 = 𝑎𝑥2 + 𝑏𝑓 𝑥2 − 𝑓 𝑥1 = 𝑎𝑥2 + 𝑏 − (𝑎𝑥1 + 𝑏)

𝐼2 − 𝐼1 = 𝑎 𝑥2 − 𝑥1⇒ 𝐼2 − 𝐼1 ∝ (𝑥2 − 𝑥1)

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ ?

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ 𝑥 − 𝑥1

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ 𝑥 − 𝑥1

Dividing them,𝐼2 − 𝐼1𝐼 − 𝐼1

=𝑥2 − 𝑥1𝑥 − 𝑥1

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


Solve for 𝐼𝐼2 − 𝐼1𝐼 − 𝐼1

=𝑥2 − 𝑥1𝑥 − 𝑥1

(𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1

= 𝐼 − 𝐼1

𝐼 = 𝐼1 + (𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


Solve for 𝐼𝐼2 − 𝐼1𝐼 − 𝐼1

=𝑥2 − 𝑥1𝑥 − 𝑥1

(𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1

= 𝐼 − 𝐼1

𝐼 = 𝐼1 + (𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼


Solve for 𝐼

𝐼 =𝐼1 𝑥2 − 𝑥1 + (𝐼2−𝐼1) 𝑥 − 𝑥1

𝑥2 − 𝑥1

𝐼 =𝐼1 𝑥2 − 𝑥 + 𝐼2 𝑥 − 𝑥1

𝑥2 − 𝑥1

𝐼 =𝐼1 𝑥2 − 𝑥

𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1

𝑥1 𝑥2

𝐼1 𝐼2

𝑥

𝐼

Example: Linear Interpolation

Solve for 𝐼

𝑥1 = 0 𝑥2 = 1

𝐼1 = 10 𝐼2 = 15

𝑥 = 0.3

𝐼 =?


Solve for 𝐼

𝐼 =𝐼1 𝑥2 − 𝑥

𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1

𝑥1 = 0 𝑥2 = 1

𝐼1 = 10 𝐼2 = 15

𝑥 = 0.3

𝐼 =?


Solve for 𝐼

𝐼 =𝐼1 𝑥2 − 𝑥

𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1

𝐼 =10(1 − 0.3)

1 − 0+15 0.3 − 0

1 − 0𝐼 = 7 + 4.5 = 11.5

𝑥1 = 0 𝑥2 = 1

𝐼1 = 10 𝐼2 = 15

𝑥 = 0.3

𝐼 =11.5

Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)𝑓 𝑄𝑖 → 𝑖𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦 𝑎𝑡 𝑄𝑖Find the value at 𝑃

https://en.wikipedia.org/wiki/Bilinear_interpolation

Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)

Find the value at 𝑃








Example

𝐼 21,14 = 162,

𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?


Example

𝐼 21,14 = 162,

𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?


Example

𝐼 21,14 = 162,

𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?


Example

𝐼 21,14 = 162,

𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?


Bilinear Interpolation5 X 8 Image

3 X 3 Image

Nearest neighbor Interpolation

Bilinear Interpolation

Nearest neighbor Interpolation


Bilinear: Alternative algorithm

• An alternative way to write the solution to the interpolation problem is

• Not linear but quadratic

Weeks 1 & 2 97

Image Interpolation:

Bicubic Interpolation

3 3

3

0 0

( , ) i j

ij

i j

f x y a x y

• The intensity value assigned to point (x,y) is obtained by the

following equation

• The sixteen coefficients are determined by using the sixteen nearest neighbors.


Bicubic Interpolation

Documents

Digital Image Processing - University of Houstonqil.uh.edu/dip/media/cosc6380/Lecture_-_4.pdfQUANTIZATION • Each pixel gray level is quantized: assigned one of a finite set of numbers