Image Representation - Heidelberg University · 18 April 2016 Representation 2 Last Time: Image Formation Incoming Light ( ray optics ) Lenses – Point Spread Functions Defocus Airy

Image Representation

Anita Sellent

18 April 2016 Representation 2

Last Time: Image Formation

● Incoming Light ( ray optics )● Lenses

– Point Spread Functions● Defocus● Airy Disk

● Sensor Setup● Sampling and Representation● Colors


Array of Integers


Mathematical Basics

● Introduction● Get experience in exercises● Get experience in applications


Basis of Vector-Spaces

● Canonical basis v∈ℝ3

v=(v1

v2

v 3)=v1(

100)+v2(

010)+v3(

001) v1, v2, v3∈ℝ

e1 e2 e3



● Canonical basis

● Other basis

v∈ℝ3

v=(v1

v2

v 3)=v1(

100)+v2(

010)+v3(

001) v1, v2, v3∈ℝ

v=a(√31784 )+b(

π21)+c (

??? )



● Canonical basis

● Other basis


Basis of a Vector Space

● Set of independent vectors

● Spans the entire vector space

a (100)+b(

010)≠(

001) ∀a ,b∈ℝ

v=(v1

v2

v 3)=v1(

100)+v2(

010)+v3(

001) ∀ v∈ℝ

3


Images as Vectors

● Canonical Basis

I=∑i=1

N

∑j=1

M

I i , j ei , j

j

i


Quiz

● How many canonical images are required to represent all N x M images?

1) 255 * ( M + N )

2) MN

3) NM

4) N*M

5) ( N * M )255

6) Don't know

j

i


Images as Vectors

● Canonical Basis

● Other Basis

I=∑i=1

N

∑j=1

M

I i , j ei , j


Images as Vectors

● Canonical Basis

● Other Basis

I=∑i=1

N

∑j=1

M

I i , j ei , j


Image Representation

● Which representations make sense?– Canonical Basis

● Easy access to pixels

– Stripes● Describe patterns (e.g. for compression )● Describe optics ( e.g. effect of point spread functions )


Literature

● Gonzales, Woods: Digital Image Processing– Chapter 4

● Weeks: Fundamentals of Electronic Image Processing– Chapter 2

● Easton: Fourier Methods in Imaging– Chapter 9 + 10


Describe Stripes

● 1D case cm( t)=cos(2 πmM

t) m∈ℤ

M-periodic

m=0


Describe Stripes


t) m∈ℤ

M-periodic

m=1


Describe Stripes


t) m∈ℤ

M-periodic

m=2


Describe Stripes


t)

M-periodic

even

f (x)=f (−x )

m=7

m∈ℤ


Describe Stripes

● 1D case sm( t)=sin(2πmM

t) m∈ℤ

M-periodic

m=2


Describe Stripes


t)

M-periodic

odd

f (x)=−f (−x)

m=7

m∈ℤ


Describe Stripes


t)

M-periodic

m=7

m∈ℤ

odd

f (x)=−f (−x)


Describe Stripes


t)

M-periodic

m=2

m∈ℤ

odd

f (x)=−f (−x)

t∈(−M2

,M2 )


Describe Stripes


t)

M-periodic

m=2

m∈ℤ

odd

f (x)=−f (−x)

t∈[0,M )


Discrete Fourier Transform

● Every finite sequence of complex numbers can be written as a sum of cosines and sines.

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

M−1

am cos (2πmM

n)+bm sin(2 πmM

n)

am , bm∈ℂ ∀m∈{0, M−1}




f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

M−1

Fm(cos (2 πmM

n)+ isin (2πmM

n))




f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

M−1

Fm(cos (2 πmM

n)+ isin (2πmM

n))Fm=∑

n=0

M−1

f n(cos(2πnM

m)−i sin(2 πnM

m))



● Every finite sequence of complex number can be written as a sum of cosines and sines.

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

M−1

Fmexp(2 π imM

n)

Fm=∑n=0

M−1

f n exp(−2π inM

m)

Euler's Formula



f n n∈{0, ... ,15 }



f n n∈{0, ... ,15 }

After optics

Bucket brigade



f n n∈{0, ... ,15 }

cm( t)=cos(2 πmM

t)

Fm=∑n=0

M−1

f n exp(−2π inM

m)



● Spatial Domain ● Frequency Domain

f n∈ℂ n∈{0, ... ,15 } Fm∈ℂ m∈{0,... ,15}

F=U fFm=∑n=0

M−1

f n exp(−2π inM

m)


High Frequencies

m = 1 m = 15


High Frequencies

m = 1 m = -1



● Spatial Domain ● Frequency Domain

f n∈ℂ n∈{0, ... ,15 } Fm∈ℂ m∈{0,... ,15}

F=U f


Computing the DFT

● Naive implementation: complexity O( N² )

● Fast Fourier Transform– M = 2

Fm=∑n=0

M−1

f n exp(−2π inM

m)

F0=∑n=0

M−1

f n

F0=f 0+ f 1 F1=f 0+ f 1(cos (−π)+ isin (−π))= f 0−f 1


Computing the DFT

● Define


w=exp(−2π i1M

) Fm=∑n=0

M−1

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1

F0=f 0+ f 1+ f 2+ f 3

F1=f 0+w f 1+w2 f 2+w3 f 3=f 0+w f 1−f 2−w f 3


Computing the DFT

● Define


w=exp(−2π i1M

) Fm=∑n=0

M−1

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1

F0=f 0+ f 1+ f 2+ f 3=( f 0+ f 2)+( f 1+ f 3)

F1=f 0+w f 1−f 2−w f 3=(f 0−f 2)+w ( f 1− f 3)

F2= f 0+w2 f 1+w4 f 2−w6 f 3=(f 0+f 2)−( f 1+ f 3)

F2= f 0+w3 f 1+w6 f 2−w9 f 3=( f 0− f 2)−w( f 1− f 3)


Computing the DFT

● Define

● Fast Fourier Transform– Complexity O( M log M )

w=exp(−2π i1M

) Fm=∑n=0

M−1

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1



● 1D Discrete Fourier Transform

● 2D Discrete Fourier Transform

f m,n∈ℂ m∈{0,... , M−1}, n∈{0, ..., N−1}

Fm=∑n=0

M−1

f n exp(−2π inM

m)

f n∈ℂ n∈{0, ... ,15 }

Fk ,l=∑m=0

M −1

∑n=0

N−1

f m,n exp (−2π i( kM

m+lN

n))


2D Discrete Fourier Transform

k = 5, l = 0 k = 0, l = 5 k = 15, l = 17



● Example: What orientation does this text have?











2D DFT

● Compute 1D DFT for each dimension

Fk ,l=∑m=0

M −1

∑n=0

N−1

f m,n exp (−2π i( kM

m+lN

n))

=∑m=0

M −1

(∑n=0

N−1

f m ,n exp (−2π ilN

n))exp (−2π ikM

m)

gm, l


Insights from (Discrete) Fourier Transform

● Sample-Densityf(t)



● Sample-Densityf(t)



cm( t)=cos(2 πmM

t)

g(t )=∑m=0

M−1

Fmexp(−2 π imM

t)


Sampling Theorem

● Connects continuous function to sample points● Works with continuous Fourier Transform

● Dirac distribution to express sampling

F (μ)=∫−∞

∞

f (t )exp(−2 πiμ t)dt

1=∫−∞

∞

δ(t)dt

f (0)=∫−∞

∞

f (t)δ( t)dt f n=f (nΔT )=∫−∞

∞

f (t)δ (t−n ΔT )dt


Tools Sampling Theorem

● 1D Fourier Transform

f (x) cos(2x)

f(x)

x

f̂( ) ( 1)( 1)2

f̂ ( )

f̂ (ξ)=F (ξ)=∫−∞

∞

f (t)cos (−2π ξ t)+i f ( t)sin(−2 πξ t)dt



● 1D Fourier Transformf̂ (ξ)=F (ξ)=∫

−∞

∞


f (x) sin(2x)

f(x)

x

f̂ ( ) i ( 1)( 1)2

f̂ ( )i



● 1D Fourier Transformf̂ (ξ)=F (ξ)=∫

−∞

∞


f (x) cos(x2)

f(x)

x

f̂ ( )( 1

4)( 1

4)

2

f̂ ( )


1D Fourier Transform Pairs

Gaussian:

Sinc:

f (x) eax2

f (x) sinc(ax) sin(x)x

f̂ ( ) 1arect

a

a 4

a 20f̂ ( )

ae

( )2

a


Ideas Sampling Theorem

● Continuous Fourier transform of sampled function sums shifted copies









● Overlapping copies prevent exact reconstruction


Sampling Theorem

● For a bandlimited function, we need to sample at least twice as often as the bandlimit to allow for exact reconstruction

|μ|>μmax→F (μ)=0

μmax<1

2ΔT

ΔT <1

2μmax


Sampling Theorem

Input Image:Sufficiently resolved by optics

Nyquist Sampling:At least 2 samples per cycle

Aliasing:Too little samples for correct reconstruction!


Downscaling

Original Size: 1344x1024 pixel

Downscale (factor 10)


Quiz

● I want to save pixels in an image. What should I do to preserve the most possible of information?

1)Throw away every 2nd pixel

2)Fourier Transform, remove frequencies, backtransform, throw away every 2nd pixel

3)Fourier Transform, remove frequencies, throw away every 2nd pixel, backtransform

4)Average pixels in neighborhood, throw away every 2nd pixel

5)Crop the image

6)I don't know


Downscaling

Downscale

Downsample



Downscaling





Downscaling





Input Image:Only large structures

Super-Sampling:More samples than necessary

Nyquist Sampling:

At least 2 samples per cycle

Downscaling


Removing Frequencies

• Remove high frequencies: low-pass-filter


Removing Frequencies

• Remove high frequencies: low-pass-filter


Anti-Aliasing

Remove fine structures via blurring

Blur Down-

sample


Convolution Theorem

● Idea: ( Weighted ) average in the spatial domain is equivalent to low-pass-filtering


Summary

● Discrete Fourier Transform provides alternative basis for images.

● Sampling theorem specifies size of details that can be reconstructed exactly


Next Week

● Exercise– On paper

– In implementation

Documents

Image Representation - Heidelberg University · 18 April 2016 Representation 2 Last Time: Image Formation Incoming Light ( ray optics ) Lenses – Point Spread Functions Defocus Airy