Mathematics and Computation in Imaging Science and Information Processing

July-December, 2003

• Organized by Institute of Mathematical Sciences and Center for Wavelet. Approximation, and Information Processing, National University of Singapore.

• Collaboration with the Wavelet Center for Ideal Data Representation.

• Co-chairmen of the organizing committee:

• Amos Ron (UW-Madison),

• Zuowei Shen (NUS),

• Chi-Wang Shu (Brown University)

Conferences

• Wavelet Theory and Applications: New Directions and Challenges, 14 - 18 July 2003

• Numerical Methods in Imaging Science and Information Processing, 15 -19 December 2003

Confirmed Plenary Speakers for Wavelet Conference

• Albert Cohen • Wolfgang Dahmen• Ingrid Daubechies • Ronald DeVore • David Donoho• Rong-Qing Jia

• Yannis Kevrekidis • Amos Ron • Peter Schröder • Gilbert Strang • Martin Vetterli

Workshops

• IMS-IDR-CWAIP Joint Workshop on Data Representation, Part I on 9 – 11, II on 22 - 24 July 2003

• Functional and harmonic analyses of wavelets and frames, 28 July - 1 Aug 2003

• Information processing for medical images, 8 - 10 September 2003

• Time-frequency analysis and applications, 22- 26 September 2003

• Mathematics in image processing, 8 - 9 December 2003

• Industrial signal processing (TBA)

• Digital watermarking (TBA)

Tutorials

• A series of tutorial sessions covering various topics in approximation and wavelet theory, computational mathematics, and their applications in image, signal and information processing.

• Each tutorial session consists of four one-hour talks designed to suit a wide range of audience of different interests.

• The tutorial sessions are part of the activities of the conference or workshop associated with.

Membership Applications

• To stay in the program longer than two weeks

• Please visit http://www.ims.nus.edu.sg

for more information

Wavelet Algorithms for High-Resolution Image Reconstruction

Zuowei Shen

Department of Mathematics

National University of Singapore

http://www.math.nus.edu.sg/~matzuows

Joint work with (accepted by SISC)

T. Chan (UCLA), R.Chan (CUHK) and L.X. Shen (WVU)

Part I: Problem Setting

Part II: Wavelet Algorithms

Outline of the talk

What is an image?

image = matrix

pixel intensity

= matrix entry

Resolution = size of the matrix

I. High-Resolution Image Reconstruction:

Resolution = 64 64 Resolution = 256 256

Four low resolution images (64 64) of the same scene.

Each shifted by sub-pixel length.

Construct a high-resolution image (256 256) from

them.

#2

#4

Boo and Bose (IJIST, 97):

#1

taking lens

CCD sensorarray

relay lenses

partially silvered mirrors

Four 2 images merged into one 4 image:

a1 a2

a3 a4

b1 b2

b3 b4

c1 c2

c3 c4

d1 d2

d3 d4

Four low resolution images

Observed high-resolution image

a1 b1 a2 b2

c1 d1 c2 d2

a3 b3 a4 b4

c3 d3 c4 d4

By permutation

Four 64 64 images merged into one by permutation:

Observed high-resolution image by

permutation

Modeling

Consider:

Low-resolution pixel

High-resolution

pixels

4

1

2

1

4

1

2

11

2

1

4

1

2

1

4

1

Observed image: HR image passing through a low-pass filter a.LR image: the down samples of observed imageat different sub-pixel position.

L f = g ,

After modeling and adding boundary condition, it can be reduced to :

Where L is blurring matrix, g is the observed image and f is the original image.

The problem L f = g is ill-conditioned.

g*1* ) ( LRLL g g*1* )( LLL

.) ( ** gf LRLL

Here R can be I, . It is called Tikhonov method ( or the least square )

Regularization is required:

Wavelet Method• Let â be the symbol of the low-pass filter. Assume:

• can be found such that

dd b, b,a ˆˆ ˆ

1ˆˆˆˆ}0\{2

2

Z

bbaa dd

• One can use unitary extension principle to obtain a set of tight frame systems.

Let be the refinable function with refinement mask a, i.e.

Let d be the dual function of :

. , 0 d

We can express the true image as

where v() are the pixel values of the high-resolution picture.

, 2 22

dvfZ

. )2( )(4 2

Z

a

The pixel values of the observed image are given by

2* , Zva

The observed function is

. )2/( )( 2

Z

dag

The problem is to find v( ) from (a * v)().

From 4 sets low resolution pixel values reconstruct f, lift

1 level up. Similarly, one can have 2 level up from 16 set...

Do it in the Fourier domain. Note that

(1) . 1ˆˆ ˆˆ}0\{2

2

Z

bbaa dd

We have

. ˆˆˆˆˆˆˆ

0\22

vvbbvaa dd

Z

or

. ˆˆˆˆˆ

0\

*22

vvbbvaa dd

Z

Generic Wavelet Algorithm:

(i) Choose ;ˆ 220 ,Lv

(ii) Iterate until convergence:

. ˆˆˆˆˆ

0\

*122

ndd

n vbbvaav

Z

Proposition Suppose that and nonzero

almost everywhere. Then for

arbitrary .

1ˆˆ0 aad

0||ˆˆ|| 2 vvn

0v̂

Regularization:

Damp the high-frequency components in the current iterant.

Wavelet Algorithm I:

(i) Choose ;ˆ 220 ,Lv

(ii) Iterate until convergence:

. ˆ ˆˆ)1(ˆˆ

0\

*122

ndd

n vbbvaav

Z

Matrix Formulation:

The Wavelet Algorithm I is the stationary iteration for

. )( gf ddd LHHLL

Different between Tikhonov and Wavelet Models:

• Ld instead of L*.

• Wavelet regularization operator.

Both penalize high-frequency components uniformly by .

Wavelet Thresholding Denoising Method:

Decompose the n-th iterate, i.e. , into different

scales: ( This gives a wavelet packet decomposition of n-

th iterate.)

nvb ˆ̂

, ˆˆˆˆˆ ˆˆˆˆˆˆˆ0,0\

1

0 22

njd

J

j

dn

Jdn vbabbavbaavb

Z

• Denoise these coefficients of the wavelet

packet by thresholding method.

nj vbab ˆˆˆˆ

Before reconstruction,

Wavelet Algorithm II:

(i) Choose ;ˆ 220 ,Lv

(ii) Iterate until convergence:

n

ddn vbbvaav ˆˆTˆˆˆ

,\

*

00

122

Z

Where T is a wavelet thresholding processing .

4 4 sensor array:

Original LR Frame Observed HR

Tikhonov Algorithm I Algorithm II

4 4 sensor array:

Tikhonov Algorithm II

SNR Tikhonov Algorithm I Algorithm II(dB) PSNR RE PSNR RE PSNR RE Iter.30 32.55 0.0437 33.82 0.0377 34.48 0.0350 940 33.88 0.0375 34.80 0.0337 35.23 0.0321 12

SNR Tikhonov Algorithm I Algorithm II(dB) PSNR RE PSNR RE PSNR RE Iter.30 29.49 0.0621 29.70 0.0601 30.11 0.0579 3040 30.17 0.0573 30.30 0.0566 30.56 0.0549 45

22 sensor array: 1 level up

44 sensor array: 2 level up

Numerical Examples:

1-D Example: Signal from Donoho’s Wavelet Toolbox.Blurred by 1-D filter.

Original Signal Observed HR Signal

Tikhonov Algorithm II

Ideal low-resolution pixel position

High-resolution

pixels

Calibration Error:

Problem no longer spatially

invariant.

Displaced low-resolution pixel

Displacement errorx

The lower pass filter is perturbed

The wavelet algorithms can be modified

Reconstruction for 4 4 Sensors: (2 level up)

Original LR Frame Observed HR

Tikhonov Wavelets

Reconstruction for 4 4 Sensors: (2 level up)

Tikhonov Wavelets

Numerical Results:

2 2 sensor array (1 level up) with calibration errors:

Least Squares Model Our Algorithm

SNR(dB) PSNR RE * PSNR RE Iterations

30 28.00 0.0734 0.0367 30.94 0.0524 2

40 28.24 0.0715 0.0353 31.16 0.0511 2

4 4 sensor array (2 level) with calibration errors:

Least Squares Model Our Algorithm

SNR(dB) PSNR RE * PSNR RE Iterations

30 24.63 0.1084 0.0492 27.80 0.0752 5

40 24.67 0.1078 0.0505 26.81 0.0751 6

(0,0)

(1,1)

(0,2)

(1,3)

(2,0)

(3,1)

(2,2)

(3,3)

(0,1) (0,3)

(1,0)

(2,1)

(1,2)

(2,3)

(3,0) (3,2)

Example: 4 4 sensor with missing frames:

Super-resolution: not enough frames

(0,1) (0,3)

(1,0)

(2,1)

(1,2)

(2,3)

(3,0) (3,2)

Example: 4 4 sensor with missing frames:

Super-resolution: not enough frames

i. Apply an interpolatory subdivision scheme to obtain the missing frames.

ii. Generate the observed high-resolution image w.

iii. Solve for the high-resolution image u.

iv. From u, generate the missing low-resolution frames.

v. Then generate a new observed high-resolution image g.

vi. Solve for the final high-resolution image f.

Super-Resolution:

Not enough low-resolution frames.

Tikhonov Algorithm I Algorithm II

PSNR RE PSNR RE PSNR RE

27.44 0.0787 27.82 0.0753 27.76 0.0758

Reconstructed Image:

Observed LR Final Solution