47
CS448f: Image Processing For Photography and Vision Sharpening

CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

  • Upload
    others

  • View
    27

  • Download
    0

Embed Size (px)

Citation preview

Page 1: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

CS448f: Image Processing For Photography and Vision

Sharpening

Page 2: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Sharpening

• Boost detail in an image without introducing noise or artifacts

• Undo blur

– due to lens aberrations

– slight misfocus

Page 3: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Recall Denoising

Input

=

Signal + Noise

Page 4: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Recall Denoising

Input

=

Signal + Noise

Page 5: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Sharpening

Input

=

Coarse + Fine

Page 6: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Sharpening

Output

=

Coarse + Fine

Page 7: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Sharpening

• Any Filter which removes fine details can be used to sharpen

– 1) Coarse = Remove Fine Details from Input

– 2) Fine = Input - Coarse

– 3) Output = Input + Fine x 0.5

• Which filters should be use to create the coarse base layer?

• What about noise?

Page 8: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Linear Sharpening Filters

• Let G be a Gaussian Kernel

– 1) Coarse = G * Input

– 2) Fine = Input - Coarse

– 3) Output = Input + Fine x 0.5

Page 9: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Convolution is Linear

• G*(a+b) = G*a + G*b

– Output = Input + 0.5 Fine

– Output = Input + 0.5 (Input - G*Input)

– Output = 1.5 Input - 0.5 G*Input

– Output = (1.5 I - 0.5 G) * Input

• Or in Fourier Space

– Output’ = (1.5 I’ - 0.5 G’) x Input’

I

Page 10: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Linear Sharpening Filters

• I is the filter that does nothing when you convolve by it, so I’ is the filter that does nothing when you multiply by it => I’ = 1

Page 11: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Linear Sharpening Filters

• The Fourier Transform of a Gaussian is a Gaussian

• G’:

Page 12: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

The result in Fourier space:

• (1.5 I’ - 0.5 G’) = amplify high frequencies

Page 13: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Demo

• ImageStack -load dog.jpg -dup -dup -dup -gaussianblur 4 -pull 1 -subtract -scale 2 -add -adjoin t -resample 10 width height -display

Page 14: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 15: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Coarse

Page 16: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Fine x 3

Page 17: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input +Fine

Page 18: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 19: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Halos:

Page 20: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Halos

Page 21: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Let’s see what Photoshop Does

• Unsharp Masking...

Page 22: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Let’s see what Photoshop Does

• Unsharp Masking creates halos

• With the threshold set, fine details are not boosted, only strong edges

Page 23: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Suggestions?

• What removes fine detail without blurring edges?

Page 24: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Median Sharpen

• The “Fine” image is the same as the “Method Noise” images in the previous lecture.

• It should only contain fine detail, not strong edges

• Let’s make the base layer with a median filter!

Page 25: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 26: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

MedianCoarse

Page 27: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

MedianFine x 3

Page 28: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

MedianResult

Page 29: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

LinearResult

Page 30: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Bilateral Sharpen

• Let’s make the base layer with a bilateral filter!

Page 31: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 32: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

BilateralCoarse

Page 33: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

BilateralFine x 3

Page 34: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

BilateralResult

Page 35: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

MedianResult

Page 36: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

LinearResult

Page 37: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Non-Local Means Sharpen?

• Non-Local Means looks for similar patches and averages my value with theirs

– Conformity with peer group

• Non-Local Means sharpening figures out what makes me different from other similar things in the image, and exaggerates that

– Rebellion against peer group

Page 38: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 39: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

NLMeansCoarse

Page 40: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

NLMeansFine x 8

Page 41: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

BilateralFine x 8

Page 42: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

NLMeansResult

Page 43: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

BilateralResult

Page 44: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Input

Page 45: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Remember...

• None of this is useful if we can’t make it go fast

Page 46: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Other Techniques

• Everyone wants to best the bilateral filter

• Two notable papers to look at:

– The Trilateral filter (Tumblin et al, EGSR 2003)

Page 47: CS448f: Image Processing For Photography and VisionCS448f: Image Processing For Photography and Vision Sharpening. Sharpening •Boost detail in an image without introducing noise

Other Techniques

• Edge Preserving Decompositions for Multi-Scale Tone and Detail Manipulation:

– Farbman et al, SIGGRAPH 2008