Image Representation - Heidelberg University · 18 April 2016 Representation 2 Last Time: Image...

Image Representation

Anita Sellent

18 April 2016 Representation 2

Last Time: Image Formation

● Incoming Light ( ray optics )● Lenses

– Point Spread Functions● Defocus● Airy Disk

● Sensor Setup● Sampling and Representation● Colors

Array of Integers

Mathematical Basics

● Introduction● Get experience in exercises● Get experience in applications

Basis of Vector-Spaces

● Canonical basis v∈ℝ3

v 3)=v1(

100)+v2(

010)+v3(

001) v1, v2, v3∈ℝ

e1 e2 e3

● Canonical basis

● Other basis

v∈ℝ3

v 3)=v1(

100)+v2(

010)+v3(

001) v1, v2, v3∈ℝ

v=a(√31784 )+b(

π21)+c (

● Canonical basis

● Other basis

Basis of a Vector Space

● Set of independent vectors

● Spans the entire vector space

a (100)+b(

010)≠(

001) ∀a ,b∈ℝ

v 3)=v1(

100)+v2(

010)+v3(

001) ∀ v∈ℝ

Images as Vectors

● Canonical Basis

I=∑i=1

∑j=1

I i , j ei , j

● How many canonical images are required to represent all N x M images?

1) 255 * ( M + N )

4) N*M

5) ( N * M )255

6) Don't know

Images as Vectors

● Canonical Basis

● Other Basis

I=∑i=1

∑j=1

I i , j ei , j

Images as Vectors

● Canonical Basis

● Other Basis

I=∑i=1

∑j=1

I i , j ei , j

Image Representation

● Which representations make sense?– Canonical Basis

● Easy access to pixels

– Stripes● Describe patterns (e.g. for compression )● Describe optics ( e.g. effect of point spread functions )

Literature

● Gonzales, Woods: Digital Image Processing– Chapter 4

● Weeks: Fundamentals of Electronic Image Processing– Chapter 2

● Easton: Fourier Methods in Imaging– Chapter 9 + 10

Describe Stripes

● 1D case cm( t)=cos(2 πmM

t) m∈ℤ

M-periodic

Describe Stripes

t) m∈ℤ

M-periodic

Describe Stripes

t) m∈ℤ

M-periodic

Describe Stripes

M-periodic

f (x)=f (−x )

m∈ℤ

Describe Stripes

● 1D case sm( t)=sin(2πmM

t) m∈ℤ

M-periodic

Describe Stripes

M-periodic

f (x)=−f (−x)

m∈ℤ

Describe Stripes

M-periodic

m∈ℤ

f (x)=−f (−x)

Describe Stripes

M-periodic

m∈ℤ

f (x)=−f (−x)

t∈(−M2

Describe Stripes

M-periodic

m∈ℤ

f (x)=−f (−x)

t∈[0,M )

Discrete Fourier Transform

● Every finite sequence of complex numbers can be written as a sum of cosines and sines.

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

am cos (2πmM

n)+bm sin(2 πmM

am , bm∈ℂ ∀m∈{0, M−1}

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

Fm(cos (2 πmM

n)+ isin (2πmM

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

Fm(cos (2 πmM

n)+ isin (2πmM

n))Fm=∑

f n(cos(2πnM

m)−i sin(2 πnM

● Every finite sequence of complex number can be written as a sum of cosines and sines.

f n∈ℂ ∀ n∈{0, M−1}

f n=1M

∑m=0

Fmexp(2 π imM

Fm=∑n=0

f n exp(−2π inM

Euler's Formula

f n n∈{0, ... ,15 }

After optics

Bucket brigade

f n n∈{0, ... ,15 }

cm( t)=cos(2 πmM

Fm=∑n=0

f n exp(−2π inM

● Spatial Domain ● Frequency Domain

f n∈ℂ n∈{0, ... ,15 } Fm∈ℂ m∈{0,... ,15}

F=U fFm=∑n=0

f n exp(−2π inM

High Frequencies

m = 1 m = 15

High Frequencies

m = 1 m = -1

● Spatial Domain ● Frequency Domain

f n∈ℂ n∈{0, ... ,15 } Fm∈ℂ m∈{0,... ,15}

Computing the DFT

● Naive implementation: complexity O( N² )

● Fast Fourier Transform– M = 2

Fm=∑n=0

f n exp(−2π inM

F0=∑n=0

F0=f 0+ f 1 F1=f 0+ f 1(cos (−π)+ isin (−π))= f 0−f 1

Computing the DFT

● Define

w=exp(−2π i1M

) Fm=∑n=0

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1

F0=f 0+ f 1+ f 2+ f 3

F1=f 0+w f 1+w2 f 2+w3 f 3=f 0+w f 1−f 2−w f 3

Computing the DFT

● Define

w=exp(−2π i1M

) Fm=∑n=0

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1

F0=f 0+ f 1+ f 2+ f 3=( f 0+ f 2)+( f 1+ f 3)

F1=f 0+w f 1−f 2−w f 3=(f 0−f 2)+w ( f 1− f 3)

F2= f 0+w2 f 1+w4 f 2−w6 f 3=(f 0+f 2)−( f 1+ f 3)

F2= f 0+w3 f 1+w6 f 2−w9 f 3=( f 0− f 2)−w( f 1− f 3)

Computing the DFT

● Define

● Fast Fourier Transform– Complexity O( M log M )

w=exp(−2π i1M

) Fm=∑n=0

f nwmn

w0=1,w2

=−1,w3=ww2

=−w ,w4=1

● 1D Discrete Fourier Transform

● 2D Discrete Fourier Transform

f m,n∈ℂ m∈{0,... , M−1}, n∈{0, ..., N−1}

Fm=∑n=0

f n exp(−2π inM

f n∈ℂ n∈{0, ... ,15 }

Fk ,l=∑m=0

M −1

∑n=0

f m,n exp (−2π i( kM

2D Discrete Fourier Transform

k = 5, l = 0 k = 0, l = 5 k = 15, l = 17

● Example: What orientation does this text have?

2D DFT

● Compute 1D DFT for each dimension

Fk ,l=∑m=0

M −1

∑n=0

f m,n exp (−2π i( kM

=∑m=0

M −1

(∑n=0

f m ,n exp (−2π ilN

n))exp (−2π ikM

Insights from (Discrete) Fourier Transform

● Sample-Densityf(t)

cm( t)=cos(2 πmM

g(t )=∑m=0

Fmexp(−2 π imM

Sampling Theorem

● Connects continuous function to sample points● Works with continuous Fourier Transform

● Dirac distribution to express sampling

F (μ)=∫−∞

f (t )exp(−2 πiμ t)dt

1=∫−∞

δ(t)dt

f (0)=∫−∞

f (t)δ( t)dt f n=f (nΔT )=∫−∞

f (t)δ (t−n ΔT )dt

Tools Sampling Theorem

● 1D Fourier Transform

f (x) cos(2x)

f̂( ) ( 1)( 1)2

f̂ ( )

f̂ (ξ)=F (ξ)=∫−∞

f (t)cos (−2π ξ t)+i f ( t)sin(−2 πξ t)dt

● 1D Fourier Transformf̂ (ξ)=F (ξ)=∫

−∞

f (x) sin(2x)

f̂ ( ) i ( 1)( 1)2

f̂ ( )i

● 1D Fourier Transformf̂ (ξ)=F (ξ)=∫

−∞

f (x) cos(x2)

f̂ ( )( 1

f̂ ( )

1D Fourier Transform Pairs

Gaussian:

f (x) eax2

f (x) sinc(ax) sin(x)x

f̂ ( ) 1arect

a 20f̂ ( )

Ideas Sampling Theorem

● Continuous Fourier transform of sampled function sums shifted copies

● Overlapping copies prevent exact reconstruction

Sampling Theorem

● For a bandlimited function, we need to sample at least twice as often as the bandlimit to allow for exact reconstruction

|μ|>μmax→F (μ)=0

μmax<1

ΔT <1

2μmax

Sampling Theorem

Input Image:Sufficiently resolved by optics

Nyquist Sampling:At least 2 samples per cycle

Aliasing:Too little samples for correct reconstruction!

Downscaling

Original Size: 1344x1024 pixel

Downscale (factor 10)

● I want to save pixels in an image. What should I do to preserve the most possible of information?

1)Throw away every 2nd pixel

2)Fourier Transform, remove frequencies, backtransform, throw away every 2nd pixel

3)Fourier Transform, remove frequencies, throw away every 2nd pixel, backtransform

4)Average pixels in neighborhood, throw away every 2nd pixel

5)Crop the image

6)I don't know

Downscaling

Downscale

Downsample

Downscaling

Input Image:Only large structures

Super-Sampling:More samples than necessary

Nyquist Sampling:

At least 2 samples per cycle

Downscaling

Removing Frequencies

• Remove high frequencies: low-pass-filter

Removing Frequencies

• Remove high frequencies: low-pass-filter

Anti-Aliasing

Remove fine structures via blurring

Blur Down-

sample

Convolution Theorem

● Idea: ( Weighted ) average in the spatial domain is equivalent to low-pass-filtering

Summary

● Discrete Fourier Transform provides alternative basis for images.

● Sampling theorem specifies size of details that can be reconstructed exactly

Next Week

● Exercise– On paper

– In implementation

Image Representation - Heidelberg University · 18 April 2016 Representation 2 Last Time: Image...

Documents

Coded Aperture Pairs for Depth from Defocus and Defocus ...€¦ · 1 Coded Aperture Pairs for Depth from Defocus and Defocus Deblurring Changyin Zhou Stephen Lin Shree K. Nayar Abstract—

Exploring Defocus Matting - MITpeople.csail.mit.edu/billf/publications/Exploring_Defocus_Matting.pdf · tant operations in image editing, photog-raphy, and film production. Matting

Image representation & description

Introduction What is “image processing and computer vision”? Image Representation

Color Image Perception, Representation and Contrast ...eeweb.poly.edu/~yao/EL6123_s16/Color_ContrastEnhancement.pdf · Color Image Perception, Representation and Contrast Enhancement

Camera Image Acquisitionand Representation

Image segmentation, Representation, and Description

Deep Depth from Defocus: how can defocus blur improve 3D

Image Segmentation Using Subspace Representation and ...vision.poly.edu/index.html/uploads/shervin_thesis.pdfvii ABSTRACT Image Segmentation Using Subspace Representation and Sparse

Defocus Magnification

Representation image

09 Image Representation

Imaging and Image Representation

FOCUSING ON OUT -OF-FOCUS: ASSESSING DEFOCUS …...FOCUSING ON OUT -OF-FOCUS: ASSESSING DEFOCUS ESTIMATION ALGORITHMS FOR THE BENEFIT OF AUTOMATED IMAGE MASKING . Geert J. Verhoeven

Medical Image Segmentation ‐‐‐‐ Module 1 · Discrete vs Continuous Image Representation 8 Discrete Image Representation + TImage Domain or Image Value can be either discrete

Image representation, sampling and quantization

PRINCIPLES AND APPLICATIONS OF DEFOCUS-IMAGE … · Defocus-image modulation processing electron microscopy 391 Scanning Microscopy Vol. 11, 1997 (Pages 391-405) 0891-7035/97$5.00+.25

Blur Image Classification based on Deep Learningdoip.buaa.edu.cn/4.pdf · blur, Gaussian blur, defocus blur and motion blur) accurately and effectively. II. METHODOLOGY A. Image Blur

Chapter 7 Case Study 1: Image Deconvolution. Different Types of Image Blur Defocus blur --- Depth of field effects Scene motion --- Objects in the scene

sparse image representation via combined transforms