SINGLE IMAGE SUPER RESOLUTIONNeeraj Kumar

11610210

INTRODUCTION

Estimate a high resolution image from a given low resolution one

Observation Model

• Blur matrix is a sparse matrix because of the localized degradation process

• Degraded image can be modeled as the convolution of original image with a finite support point spread function

LR Image Generation- A diffusion perspective

LR Generation- An illustration

Inverse Diffusion

We seek a general inverse mapping from LR to HR pixels!

Background

Proposed Algorithm

Zero-phase component analysis

Raw patches PCA with 90% variance ZCA patches

Proposed Algorithm (SR PNN)• Input: Data Available M x N LR image• Output: Desired sM x sN HR image, s being SR factor

1. Generate an image LR’ (low resolution approximation of input LR image) by applying blurring kernel and downsampling according to LR generation process to given LR image.

2. Extract vectorised, n x n (n being odd) patches from LR’ with one pixel over-lap and corresponding vectorised high resolution child pixels from given LR image.

3. Apply ZCA whitening on vectorised LR’ patches and LR pixels.

4. Train a polynomial neural network using GMDH type algorithm, to learn an inverse mapping g(.) from parent LR’ patches (ZCA whitened) to child LR pixels (ZCA whitened).

5. Extract vectorised, n x n (n being odd) patches from given LR image with one pixel over-lap and apply ZCA whitening.

6. Generate desired HR pixels using vectorised patches of step 5 as input to trained polynomial neural network of step 4.

7. After undoing ZCA whitening, re-arrange vectorised HR pixels at their respective locations and return reconstructed HR image.

Experimental Results

Experimental Results

Experimental Results (2x)

Experimental Results (4x)

Experimental Results

Experimental Results

Effect of Window Size

Generalization Performance

Computational Cost

Wavelet based SR

• Presented in previous progress seminar• Algorithm 1- Detail to Detail coefficient prediction• Algorithm 2- Approximation to Detail coefficient prediction• Algorithm 3- Combine Algorithm 1 and 2

• Submitted in Transactions on Circuits and Systems for Video Technology• Response- Major review

Wavelet Based SR

On reducing NMF admissible solutions

On reducing NMF admissible solutions

On reducing NMF admissible solutions- Proposed Approach

OS-SNMF

On reducing NMF admissible solutions- Proposed Approach

Experimental Results

Future Work

• Revise TCSVT paper• Medical Image SR

• Application of the developed techniques on optical microscopy• Abhishek Vahadane will co-ordinate the project

• Work on other inverse problems• Reducing the set of admissible solutions of non-negative matrix

factorization• Sparsified NMF