Image Denoising with K-SVD

Priyam Chatterjee

EE 264 – Image Processing & ReconstructionInstructor : Prof. Peyman MilanfarSpring 2007

2

Sparseland Model

Defined as a set {D,X,Y} such that

DY t X

Figure courtesy Michael Elad

3

Sparse Coding

Given a D and yi, how to find xi

Constraint : xi is sufficiently sparse

Finding exact solution difficult

Approximate solution good enough ?

4

Orthogonal Matching Pursuit

Select dk with maxprojection on residue

xk = arg min ||y-Dkxk||

Update residue

r = y - Dkxk

Check terminating condition

D, y x

5

OMP : features

Greedy algorithm

Can find approximate solution

Close solution if T is small enough

Simplistic in nature

6

Dictionary Selection What D to use ?

A fixed overcomplete set of basis : Steerable wavelet Contourlet DCT Basis ….

Data Adaptive Dictionary – learn from data

7

K-SVD Algorithm

Select atoms from input

Atoms can be patches from the image

Patches are overlapping

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

8

K-SVD Algorithm

Use OMP or any other fast method

Output gives sparse code for all signals

Minimize error in representation

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

9

K-SVD Algorithm

Replace unused atom with minimally represented signal

Identify signals that use k-th atom (non zero entries in rows of X)

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

10

K-SVD Algorithm

Deselect k-th atom from dictionary

Find coding error matrix of these signals

Minimize this error matrix with rank-1 approx from SVD

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

11

K-SVD Algorithm

[U,S,V] = svd(Ek)

Replace coeff of atom dk in X with entries of s1v1

dk = u1/||u1||2

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

12

Denoising framework

A cost function for : Y = Z + n

Solve forPrior term

13

Denoising Framework

Break problem into smaller problems

Aim at minimization at the patch level

Select i-th patch of Z accounted for

implicitly by OMP

14

Denoising Framework

Solution :

Denoising by normalized weighted averaging

Initialize Dictionary

Sparse Coding(OMP)

Update Dictionary

One atom at a time

Averaging of patches

15

Proof of the pudding – low noise

Denoising under presence of AWGN of std. dev 10

PSNR 28.12 dB PSNR 34.16 dB

16

High noise case – std dev 50

PSNR 14.75 dB PSNR 24.93 dB

17

Outside the math :

Similar atoms in dictionary should be replaced with signals that are least represented

Atoms which are least used should be replaced by signals that are least represented