Image sampling and image quantization - DUTH · Digital image processing Chapter 3. Image sampling and quantization IMAGE SAMPLING AND IMAGE QUANTIZATION 1. Introduction 2. Sampling

Digital image processing Chapter 3. Image sampling and quantization

IMAGE SAMPLING AND IMAGE QUANTIZATION

1. Introduction

2. Sampling in the two-dimensional space

Basics on image sampling

The concept of spatial frequencies

Images of limited bandwidth

Two-dimensional sampling

Image reconstruction from its samples

The Nyquist rate. The alias effect and spectral replicas superposition

The sampling theorem in the two-dimensional case

Non-rectangular sampling grids and interlaced sampling

The optimal sampling

Practical limitations in sampling and reconstruction

3. Image quantization

4. The optimal quantizer

The uniform quantizer

5. Visual quantization

Contrast quantization

Pseudo-random noise quantization

Halftone image generation

Color image quantization

1. Introduction

f(x,y) fs(x,y) u(m,n)

u(m,n)

Sampling Quantization Computer

Computer D/A

conversion Display

Digitization

Analog display

Fig 1 Image sampling and quantization / Analog image display


2. Sampling in the two-dimensional space

Basics on image sampling


x

y

f(x,y)

The concept of spatial frequencies

- Grey scale images can be seen as a 2-D generalization of time-varying signals (both in the analog

and in the digital case); the following equivalence applies:


1-D signal (time varying) 2-D signal (grey scale image)

Time coordinate t Space coordinates x,y

Instantaneous value: f(t) Brightness level, point-wise: f(x,y)

A 1-D signal that doesn’t vary in time (is constant)

= has 0 A.C. component, and only a D.C.

component

A perfectly uniform image (it has the same

brightness in all spatial locations); the D.C.

component = the brightness in any point

The frequency content of a 1-D signal is

proportional to the speed of variation of its

instantaneous value in time:

νmax ~ max(df/dt)

The frequency content of an image (2-D signal) is

proportional to the speed of variation of its

instantaneous value in space:

νmax,x ~ max(df/dx); νmax,y ~ max(df/dy)

=> νmax,x , νmax,y = “spatial frequencies”

Discrete 1-D signal: described by its samples => a

vector: u=[u(0) u(1) … u(N-1)], N samples; the

position of the sample = the discrete time moment

Discrete image (2-D signal): described by its

samples, but in 2-D => a matrix: U[M×N],

U={u(m,n)}, m=0,1,…,M-1; n=0,1,…,N-1.

The spectrum of the time varying signal = the real

part of the Fourier transform of the signal, F(ω);

ω=2πν.

The spectrum of the image = real part of the

Fourier transform of the image = 2-D

generalization of 1-D Fourier transform, F(ωx,ωy)

ωx=2πνx; ωy=2πνy

Images of limited bandwidth

• Limited bandwidth image = 2-D signal with finite spectral support:

F(νx, νy) = the Fourier transform of the image:

The Fourier transform of the The spectral support region

limited spectrum image

|F(νx, νy)|

-νx0

νx0

νx0

νy0

-νy0 -νy0

νy

νy

νx νx

The spectrum of a limited bandwidth image and its spectral support


.,,,2

dxdyeyxfdxdyeeyxfF

yxjyjxjyx

yxyx

00 ||,||,0),( yyxxyxF

Two-dimensional sampling (1)

m nyxs ynyxmxynxmfyxgyxfyxf ),(),(),(),(),( ),(


The common sampling grid = the uniformly spaced, rectangular grid:

y

x y

x

m nyx ynyxmxyxg ),(),(),(

Image sampling = read from the original, spatially continuous, brightness function f(x,y), only in the black

dots positions ( only where the grid allows):

.,

,,0

,),,(),(

Z

nm

otherwise

ynyxmxyxfyxfs

• Question: How to choose the values Δx, Δy to achieve:

-the representation of the digital image by the min. number of samples,

-at (ideally) no loss of information?

(I. e.: for a perfectly uniform image, only 1 sample is enough to completely represent the image =>

sampling can be done with very large steps; on the opposite – if the brightness varies very sharply => very

many samples needed)

The sampling intervals Δx, Δy needed to have no loss of information depend on the spatial frequency

content of the image.

Sampling conditions for no information loss – derived by examining the spectrum of the image by

performing the Fourier analysis:

The sampling grid function g(Δx, Δy) is periodical with period (Δx, Δy) => can be expressed by its Fourier

series expansion:


.),(),(),(),(

:ansformFourier tr),(),(),(

22),(

22

),(

dxdyeeyxgyxfdxdyeeyxfF

yxgyxfyxf

yyjxxjyx

yyjxxjSyxS

yxs


x y yy

ljx

x

kj

yx

k l

yy

ljx

x

kj

yx

dxdyeeyxgyx

lka

eelkayxg

0 0

22

),(

22

),(

.),(11

),(

:where

,),(),(

Since:

Therefore the Fourier transform of fS is:

The spectrum of the sampled image = the collection of an infinite number of scaled spectral replicas

of the spectrum of the original image, centered at multiples of spatial frequencies 1/Δx, 1/ Δy.


.,

1),(

),(1

),(

),(1

),(

1),(),(

22

22

22

22

k lyxyxS

k l

y

lyyj

x

kxxj

yxS

y

lyyj

x

kxxj

k lyxS

yyjxxjy

ylj

x

xkj

k lyxS

y

l

x

kF

yxF

dxdyeeyxfyx

F

dxdyeeyxfyx

F

dxdyeeeeyx

yxfF


.;,,11

),(

,,0

0,1),(),;0[);0[),(for ),(

lkyx

lka

otherwise

yxifyxgyxyx yx


Original image Original image spectrum – 3D Original image spectrum – 2D

2-D rectangular sampling grid

Sampled image spectrum – 3D Sampled image spectrum – 2D

Fig.4 The sampled image spectrum

xs xs-x0

2x0

x0

1/y

1/x

2y0 y0

ys-y0

xs

3

1

2

ys

x

y

Image reconstruction from its samples

00 2,2 yysxxs 00 2

1,

2

1

yx

yx

otherwise ,0

),(,)(

1

),(yx

ysxsyxH

m n

ynyxmxhynxmsfyxf

yxsfyxhyxf

yxFyxsFyxHyxF

,,,~

,,,~

),(),(),(),(~


Let us assume that the filtering region R is rectangular, at the middle distance between two spectral

replicas:

ysy

ysy

xsx

xsxyxh

ysy

xsx

ysxsyxH

sinsin

,

otherwise ,0

2 and

2,

)(

1

),(

nysy

nysy

mxsx

mxsx

m n

ynxmsf

m n

ynyxmxhynxmsfyxf

sinsin

,,,,~


Since the sinc function has infinite extent => it is impossible to implement in practice the ideal LPF

it is impossible to reconstruct in practice an image from its samples without error if we sample it

at the Nyquist rates.

Practical solution: sample the image at higher spatial frequencies + implement a real LPF (as close to

the ideal as possible).

a

aanysymxsx

m n

ynxmsfyxf

sin

sinc where,sincsinc,,~

1-D sinc function 2-D sinc function

The Nyquist rate. The aliasing. The fold-over frequencies

Note: Aliasing may also appear in the reconstruction process,

due to the imperfections of the filter!

How to avoid aliasing if cannot increase the sampling frequencies?

By a LPF on the image applied prior to sampling!

00 2,2 yysxxs

Fig. 5 Aliasing – fold-over frequencies

0

y0

y

2x0

x

2y0

x0 0


The Moire effect

“Jagged” boundaries

Non-rectangular sampling grids. Interlaced sampling grids

a) Image spectrum

1 2/

F(νx, νy)=1

νx

νy

1/2

1/2

-1/2

-1/2

b) Rectangular grid G1

n

m -1 3 2 1 0 1

1

c) Interlaced grid G2

n

2

1 2 2

1 -1 -2 2 0 m

d) The spectrum using G1

x

x x

x

1

-1 1 0

2

1

e) The spectrum using G2

1

2

Interlaced sampling


f x y am n m n

m n

( , ) , ,

,

0

Optimal sampling = Karhunen-Loeve expansion:

Image reconstruction from its samples in the real case

The question is: what to fill in the “interpolated” (new) dots?

Several interpolation methods are available; ideally – sinc function in the spatial domain; in

practice – simpler interpolation methods (i.e. approximations of LPFs).


The 1-D

interpolation

function

Graphical

representation

p(x)

The 2-D

interpolation

function

pa(x,y)=p(x)p(y)

Frequency

response

pa(1,2)

pa(1,0)

Rectangular

(zero-order

filter)

p0(x)

0 x

1/x

x/2 -x/2

1

xrect

x

x

p x p y0 0( ) ( )

sinc sinc

1

0

2

02 2x y

4x0

0

1

x

Triangular

(first order

filter)

p1(x)

x -x 0

x

1/x

1

xtri

x

x

p x p x0 0( ) ( )

p x p y1 1( ) ( )

sinc sinc

1

0

2

0

2

2 2x y

4x0

0

1

x

n-order filter

n=2, quadratic

n=3, cubic spline

pn(x)

0 x

p x p x

n

0 0( ) ( )

convolu@ii

p x p yn n( ) ( )

sinc sinc

1

0

2

0

1

2 2x y

n

4x0

0

1

x

Gaussian

pg(x) 2

0x

1

2 22

2

2

exp

x

1

2 22

2 2

2 exp

( )

x y

exp ( ) 2 2 2

1

2

2

2

0

1

x

Sinc

2x

0 x

1

x

x

xsinc

1

x y

x

x

y

ysinc sinc

rect rectx y

1

0

2

02 2

1

2x0

0

Image interpolation filters:


Image interpolation examples:

1. Rectangular (zero-order) filter, or nearest neighbour filter, or box filter:


0x

1/x

x/2-x/2

Original Sampled Reconstructed


2. Triangular (first-order) filter, or bilinear filter, or tent filter:



x-x0

x

1/x


3. Cubic interpolation filter, or bicubic filter – begins to better approximate the sinc function:



2x

0 x

Practical limitations in image sampling and reconstruction

Fig. 7 The block diagram of a real sampler & reconstruction (display) system

Analog display

pa(-x,-y)

Ideal sampler

x,y

Scanning

system

aperture

ps(-x,-y)

Input

image

Real scanner model

g~(x,y) gs(x,y) g(x,y)

Fig. 8 The real effect of the interpolation

Interpolation filter or

display system spectrum

1 xs/2 -xs/2

Pa(1,0)

0

Spectral losses

Interpolation error

Input image spectrum Sampled

image

spectrum Reconstructed

image spectrum

-

0

-xs/2 xs/2


3. Image quantization

Fig. 9 The quantizer’s transfer function

Quantizer

t1

Quantizer’s

output u’ u

tk tL+1 t2

rL

rk

r2

r1

Quantization

error

tk+1


3.1. Overview

3.2. The uniform quantizer

The quantizer’s design:

• Denote the input brightness range:

• Let B – the number of bits of the quantizer => L=2B reconstruction levels

• The expressions of the decision levels:

• The expressions of the reconstruction levels:

• Computation of the quantization error: for a given image of size M×N pixels,

U – non-quantized, and U’ – quantized => we estimate the MSE:

22

1 qktkr

ktktkr

t t t t qk k k k 1 1 constant

L

minlMaxLqqktkt

L

tLtq

1,11


MaxLLtlt 1;min1

MaxLminlu ;

E.g. B=2 => L=4

t1=0 t2=64 t3=128 t4=192 t5=256

r1=32

r2=96

r3=160

r4=224

Uniform quantizer transfer function

Decision levels

Reconstr

uction levels

L

k

t

t

duuUlinhkruM

m

N

n

nmunmuMN

k

k1

)(,2)(

1

0

1

0

2,',1 1

Examples of uniform quantization and the resulting errors:


B=1 => L=2

t1=0 t2=128 t3=256

r1=64

r2=192


Decision levels

Reconstr

uction levels

Non-quantized image Quantized image

Quantization error; MSE=36.2

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

The histogram of the non-quantized image



B=2 => L=4 Non-quantized image Quantized image

Quantization error; MSE=15


t1=0 t2=64 t3=128 t4=192 t5=256

r1=32

r2=96

r3=160

r4=224


Decision levels

Reconstr

uction levels

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000



B=3 => L=8; false contours present Non-quantized image Quantized image

Quantization error; MSE=7.33


t1=0 t2=32 t3=64 t4=96 t5=128 t6=160 t7=192 t8=224 t9=256

r1=16

r2=48

r3=80

r4=112

r5=144

r6=176

r7=208

r8=240


Decision levels

Reconstr

uction levels

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

3.2. The optimal (MSE) quantizer (the Lloyd-Max quantizer)

1L

1

t

t

u22 (u)duh)u'(u])u'E[(ue

L

i

t

t

ui

i

i

duuhru1

21

)()(

Lkduuhrur

thrtrtt

k

k

t

t

uk

k

kukkkk

k

10)()(2

0)()()(

1

221

tr r

k

k k 1

2 kt

t

u

t

t

u

k u|uE

(u)duh

(u)duuh

r1k

k

1k

k

p u p t t t t t u tu u j j j j j j( ) ( ), ( ),

1

21 1


1

3/1

3/1

11

1

1

1

)]([

)]([

t

duuh

duuhA

tL

k

t

t

u

tz

t

u

k

tj+1

pu(u)

u

t2 tL+1tjt1

(Gaussian) , or (Laplacian)

( variance , - mean)

2

2

2 2

)(exp

2

1)(

uuhu

uuhu exp

2)(

22

3

3/1

2

1

1

)]([12

1

Lt

t

u duuhL


Examples of optimal quantization and the quantization error:

B=1 => L=2 Non-quantized image Quantized image

The quantization error;

MSE=19.5

The non-quantized image histogram

t1=0 t2=89 t3=256

r1=24

r2=153

Functia de transfer a cuantizorului optimal

Nivelele de decizie

Niv

ele

le d

e r

econstr

uctie

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

The evolution of MSE

in the optimization, starting

from the uniform quantizer

1 2 3 4 5 6 7 818

20

22

24

26

28

30

32

34

36

38


B=2 => L=4


MSE=9.6

t1=0 t2=68 t3=136 t4=169 t5=256

r1=20

r2=115

r3=156

r4=181


Nivelele de decizie

Niv

ele

le d

e r

econstr

uctie

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4 5 6 7 8 9 109

10

11

12

13

14

15








B=3 => L=8


MSE=5

t1=0 t2=34 t3=78 t4=113 t5=136 t6=156t7=173 t8=203 t9=256

r1=14

r2=54

r3=101

r4=125

r5=147

r6=165

r7=181

r8=224


Nivelele de decizie

Niv

ele

le d

e r

econstr

uctie

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

0 2 4 6 8 10 12 144.5

5

5.5

6

6.5

7

7.5








3.3. The uniform quantizer = the optimal quantizer for the uniform grey level

distribution:

otherwise

tutttuh

L

Lu

0

,1

)(11

11

rt t

t t

t tk

k k

k k

k k

( )

( )

1

2 2

1

1

2 2

tt t

k

k k

1 1

2t t t t qk k k k 1 1 constant

qt t

Lt t q r t

qL

k k k k

1 1

1 2, ,

1

122

2

2 2

qu du

q

q

q

/

/

dBB6210logSNR,2 2B

10

2B

2

u

therefore


3.4. Visual quantization methods

• In general – if B<6 (uniform quantization) or B<5 (optimal quantization) => the "contouring"

effect (i.e. false contours) appears in the quantized image.

• The false contours (“contouring”) = groups of neighbor pixels quantized to the same value <=>

regions of constant gray levels; the boundaries of these regions are the false contours.

• The false contours do not contribute significantly to the MSE, but are very disturbing for the

human eye => it is important to reduce the visibility of the quantization error, not only the MSQE.

Solutions: visual quantization schemes, to hold quantization error below the level of visibility.

Two main schemes: (a) contrast quantization; (b) pseudo-random noise quantization


Uniform quantization, B=4 Uniform quantization, B=6 Optimal quantization, B=4

3.4. Visual quantization methods

a. Contrast quantization

• The visual perception of the luminance is non-linear, but the visual perception of contrast is linear

uniform quantization of the contrast is better than uniform quantization of the brightness

contrast = ratio between the lightest and the darkest brightness in the spatial region

just noticeable changes in contrast: 2% => 50 quantization levels needed 6 bits needed with a

uniform quantizer (or 4-5 bits needed with an optimal quantizer)

f

-1()

contrast -

brightness

u’ c u

c’

Brightness

f()

brightness-

contrast

MMSE

quantizer

3/1 ;1 typ.;

or

)1ln(/ ,18...6 typ.;10),1ln(

uc

uuc


Examples of contrast quantization:

• For c=u1/3:


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

t1=0t2=3.9844 t3=31.875 t4=107.5781 t5=2550

50

100

150

200

250

The transfer function of the contrast quantizer

Decision levels

Reconstr

uction levels

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

Examples of contrast quantization:

• For the log transform:


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

t1=0 t2=46.0273 t3=102.2733 t4=171.0068 t5=2550

50

100

150

200

250

The transfer function of the contrast quantizer

Decision levelsR

econstr

uction levels

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

b. Pseudorandom noise quantization (“dither”)

K bits

quantizer

Uniformly distributed

pseudorandom noise,

[-A,A]

- + +

+

u’(m,n) u(m,n) v(m,n) v’(m,n)

(m,n)


Uniform quantization, B=4

Large dither amplitude

Small dither amplitude Prior to dither subtraction

Fig. 13

a. 3 bits quantizer =>visible false contours;

b. 8 bits image, with pseudo-random noise added in the range [-16,16]; c. the image from Figure b) quantized with a 3 bits quantizer

d. the result of subtracting the pseudo-random noise from the image in

Figure c)

a b

c d


Halftone images generation

Fig.14 Digital generation of halftone images

Pseudorandom matrix

0(m,n)A

0u(m,n)A

Halftone

display

A

+ +

+ v’(m,n)

Thresholding v’

v(m,n) Luminance

H1

40 60 150 90 10

80 170 240 200 110

140 210 250 220 130

120 190 230 180 70

20 100 160 50 30

H2

52 44 36 124 132 140 148 156

60 4 28 116 200 228 236 164

68 12 20 108 212 252 244 172

76 84 92 100 204 196 188 180

132 140 148 156 52 44 36 124

200 228 236 164 60 4 28 116

212 252 244 172 68 12 20 108

204 196 188 180 76 84 92 100

Fig.15 Halftone matrices


Demo: http://markschulze.net/halftone/index.html

http://markschulze.net/halftone/index.html

Fig.3.16


Color images quantization

Fig.17 Color images quantization

Quantizer

Quantizer

Quantizer

Color space

inverse

transformation

Color space

transformation

T1’ T1

T2 T2’

T3’ BN’

GN’

RN’

BN

GN

RN

T3


Documents

Image sampling and image quantization - DUTH · Digital image processing Chapter 3. Image sampling and quantization IMAGE SAMPLING AND IMAGE QUANTIZATION 1. Introduction 2. Sampling