Combining the Power of Internal & External Denoising

Inbar MosseriThe Weizmann Institute of Science , ISRAELICCP , 2013

Outline Introduction Background Patch_psnr Results

Internal Denoising

NLM BM3D

Denoising using other noisy patches within the same noisy image

External Denoising

EPLL Sparse

Denoising using external clean natural patches or a compact representation

a) Original b) Noisy input c) Internal NLM d) External NLMe) Combinining (c)&(d)

Internal vs. External Denoising

Internal vs. External Patch Preference

the higher the noise in the image, the stronger the preference for internal denoising

var( )atchSNR(p ) var( )def

npPn

PatchSNR

PatchSNRpatches with low PatchSNR (e.g., insmooth image regions) tend to prefer Internal denoising

patches with high PatchSNR (edges, texture) tend to prefer External denoising

Overfitting the Noise-Mean

𝑝𝑛=𝑝+𝑛=�̂�+�̂��̂�=𝑝+𝑛�̂�=𝑛−𝑛

𝑅𝑀𝑆𝐸 𝑖𝑑𝑒𝑎𝑙=√𝐸𝑛 (𝑛2 )=√∫𝑛2𝑃𝑟(𝑛)𝑑𝑛=𝜎√𝑑

d : patch size ~ N(0, )

the empirical mean/variance of the noise within an individual small patch is usually not zero/σ2

Fitting of the Noise Mean

The denoising error grows linearly with the deviation from zero of the empirical noise-mean within the patch. In contrast, the denoising error is independent of the empirical noise variance within the patch.

For smooth patches with low var(p), after removing the mean , pn = p + n .

These patches are dominated by noise.There are high correlation between a random noise patch n and its similar natural patch NN(n)

Overfit the Noise Detail

Overfit the Noise Detail

For smooth patches with low var(p), after removing the mean , pn = p + n .

These patches are dominated by noise.There are high correlation between a random noise patch n and its similar natural patch NN(n)

Estimate the PatchSNR var( )var( )atchSNR(p ) 1var( ) var( )

nn

ppPn n

But var(n) is also unknown and patch-dependent.

we approximate var(n) using one of the existing denoising algorithm and get the denoised version of , so：