ELE 488 Fall 2006 Image Processing and Transmission (10-12-06)

ELE 488 F06

ELE 488 Fall 2006Image Processing and Transmission

(10-12-06)

Re-sampling and Re-sizing

1D 2D, sinc interpolation nearest neighbor, bilinear, bicubic, . . .

Geometric Transformation

translation rotation scaling Affine

10/12/06

ELE 488 F06

Integer Subsampling by M (=2)

Lower sampling rate Aliasing?

ELE 488 F06

Gaussian Blur (.5,1) followed by subsampling by 2 ,4

subsampling by 2

subsampling by 4

ELE 488 F06

1 D – Decreasing Sampling Rate by M +----+ x(n)--->| ↓ M |---> y(n)=x(nM) +----+

Y(ω) =M

1

1M

0rX(

M

r2πω ). M=2

ELE 488 F06

1 D – Increasing Sampling Rate by L

Use LPF (stopband edge ωc = π/M) to remove “image”.

+----+ x(n)--->| ↑L |---> y(nL)=x(n) +----+ y(Ln+k)=0 for 0<k<L Y(ω) = X(ωL). X(ω) ‘compressed’ by L.

+----+ +-----+ x(n)--->| ↑L |------->| LPF |---> v(n) +----+ y(n) +-----+

ELE 488 F06

2D Sampling – Spectrum of Sampled SignalAliasing

ELE 488 F06

Resizing from M x N down to I x J

I x J gridM x N grid

x

y

Image value at red pixels known. To determine value at green pixels.

(Interpolation)

MΔs = I Δss, NΔs = J Δss

ELE 488 F06

2-D Interpolation Problem

Image value at red pixels knownTo determine value at green pixels Interpolation

In Theory – sinc interpolation at new grid points, NOT practical

Approximate reconstruction with reduced computation

f (x,y) = nm,

f (mΔ,nΔ) sinc(x/ Δ – m) sinc(y/ Δ – n)

f ss(i,j ) = f (iΔss,jΔss) = nm,

f s(m,n) sinc(iΔss/ Δs – m) sinc(jΔss/ Δs – n)

ELE 488 F06

Nearest Neighbor Interpolation

(x,y) = (iΔss, -jΔss)

(x/ Δs, y/ Δs) = (iΔss/ Δs, –jΔss/ Δs) = (i M/ I , -jN/ J )

(mc, nc) = ( round (i M/ I ), - round (jN/ J ) )

(mc,nc): closest red pixel to

(x,y), to be interpolated

Simple,Fast,but…

Note: indexing from 0

f ss(i,j ) = f s (mc, nc)

ELE 488 F06

Bilinear Interpolation

Use 4 nearest neighborsof (i,j) to interpolate the image value at (x,y)

ELE 488 F06

Bilinear Interpolation: final equations

More computation than nearest neighborBetter accuracyStill fast

More complex interpolations

ELE 488 F06

Resizing from I x J up to M x N

I x J gridM x N grid

MΔs = I Δss, NΔs = J Δss

ELE 488 F06

2-D Interpolation Problem

Given image value at green pixels, to determine image value at red pixels.

Same 2-D interpolation problem. Use same interpolation methods: sinc, nearest neighbor, bilinear, . . .

ELE 488 F06

Example

nearestneighbor

bilinear

ELE 488 F06

Reduced Complexity Interpolation only an Approximation

If possible, always interpolate directly from the original image.

original

First interpolation

Second interpolation

ELE 488 F06

From a Hotel Window

ELE 488 F06

Lense Distortion

ELE 488 F06

Geometric Distortion

ELE 488 F06

Images related by a geometric transformation

ELE 488 F06

Registration by manual scaling and translation

ELE 488 F06

Photos: P. Ramadge

Spatially warp one image so that it best matches the other image.

What is the correct warp?

ELE 488 F06

Photos: Ingrid Daubechies, Mosaic: R. Radke –5/01

What is the correct warp?

ELE 488 F06

Registration Using Affine Transformation

translation, rotation, scaling, shearing

ELE 488 F06

Registration Using Projective Transformation

Add: tilting

ELE 488 F06

Geometric Image Transformations

Rotation

Scale

Polynomial

Affine

Shear

Projective

2D image 2D image

ELE 488 F06

Interpolation

From red pixels to green pixels

Have discussed:

Sinc interpolation Nearest Neighbor Bilinear

ELE 488 F06

Geometric Mapping

• Two cartesian coordinate systems: (x,y) and (x’,y’)• Forward Mapping (x’,y’) = h(x,y)

– Map locations on input image plane to output image plane

• Reverse or Inverse Mapping (x,y) = h-1(x’,y’)– Map locations on the output image plane back onto the

input image plane

x

yx’

y’

ELE 488 F06

Translation (Rigid Body Transformations)

Translate image by

x

y

x’

y’

Translation changes the origin of the Cartesian coordinate system, not the appearance of the image.

Translation preserves length & angle..

ELE 488 F06

Rotation (Rigid Body Transformations)

Transformation preserves length & angle.

Rotate image counterclockwise by

x’

y’

(x, y)

(x’, y’)

ELE 488 F06

Coordinate System on the Input Image

w = (x, y)T coordinates of pixel at (m,n) (x,y) ↔ (m,n)

wt: x-y coordinates of pixel (0,0)

Inverting this expression

x

y

(m,n)

m

n(0,0)

Index from 0!coordinates in the sample grid for

the given w and wt

Not necessarily integers

ELE 488 F06

Coordinate System on the Output Image

w’ = (x’, y’)T coordinates of pixel at (m’,n’)

w’t: x’-y’ coordinates of pixel (0,0)

Inverting this expression

Index from 0!

Given w’ and wt’, this is the

(non integer) coordinates in the sample grid

x’

y’

(m’,n’)

m’

n’(0,0)

ELE 488 F06

Inverse Mapping of the Pixel Values

x

yx’

y’

(m’,n’)w

Start from the grid coordinates of each pixel in the output image, find the cartesian coordinates w’, map these back onto input image cartesian coordinates w, then find the noninteger grid coordinates of this point on the input image. (Not necessarily integers need to interpolate)

ELE 488 F06

Final Step: Interpolation

Use NN, bilinear or … interpolation to estimate the value of the input image at the indicated point.This is the value at the (m’,n’) pixel in the output image.

ELE 488 F06

Special Case: Affine Transformation

Linear transformation and translation: h(w) = Hw + z = w’

Hence:

translation of origin of the x-y coordinate system from the (0,0) pixel in units of pixels.

translation of the origin of the x’-y’ coordinate system from the (0,0) pixel in units of pixels.

translation introduced by h in units of pixels.

w = h–1(w’) = H–1(w’ – z)

ELE 488 F06

Affine Transformation

Note: By measuring all translations in units of pixels, Δ no longer appears explicitly in the equation.

ELE 488 F06

Example: Rotation about the image center

x

y

x’

y’

(m’,n’)

(m,n)

?

ELE 488 F06

Rotation example continued

Rotation is an affine transformation with no translation term.

For rotation about the image center:

and:

ELE 488 F06

ScalingUniform scaling:

(preserve angle and shape)

Differential scaling:

Scaling is an affine transformation.

ELE 488 F06

Example of 3rd Order Polynomial Warpingclear; clfP=[0 1; -1 0]; Pinv=[0 -1; 1 0];I=im2double(imread('apple.jpg'));[M,N,P]=size(I); D=2/M; L=100; wt=D*[-(N+1)/2 (M+1)/2]'; wpt=D*[-(L+1) (L+1)]';Mp=2*L+1;Np=2*L+1;a=[0 0.5 0 0.3]; b=[0 0 0.5 0.3];for i=1:1:Mp for j=1:1:Np wp=D*P*[i-1 j-1]' + wpt; w(1)=a(1)+a(2)*wp(1)+a(3)*wp(2)+a(4)*wp(1)^3; w(2)=b(1)+b(2)*wp(1)+b(3)*wp(2)+b(4)*wp(2)^3; kh=Pinv*(w'-wt)/D + [1 1]'; [m n]=round(kh); % nearest neighbor interp. if (m>=1)&(m<=M)&(n>=1)&(n<=N) oJ(i,j,:)=I(m,n,:); else oJ(i,j,:)=[0.25 0.25 0.25]; end endendfigure(2); imshow(oJ); Title(‘Transformed Image’)

ELE 488 F06

Example of Polynomial

Spatial Warping

Images: Pratt, Digital Image Processing, 2nd Ed.

ELE 488 F06

Δ – a scaling factor

In general (not necessarily affine),

where

• Δ disappears from the expression at the expense of selecting a different warping function. • For affine h, Δ does not play a role. • For nonlinear h it is a convenient scaling factor – selecting an appropriate value for Δ can simplify the selection of h.