20050430(Multiscale Image Sharpening Adaptive to Edge Profile)

8/3/2019 20050430(Multiscale Image Sharpening Adaptive to Edge Profile)

1/41

School of Electrical Engineering and Computer Science

Kyungpook National Univ.

MultiscaleMultiscale image sharpeningimage sharpening

adaptive to edge profileadaptive to edge profile

Journal of Electronic Imaging,Journal of Electronic Imaging, vvol. 14, no. 1,ol. 14, no. 1,Jan.Jan.--Mar. 2004Mar. 2004

Hiroaki Kotera and Hui Wang


2/41

2 / 3

New image sharpening method

Properties

Adaptation to the local edge slopes Suppression of background noises

Method

Transforming RGB to YIQ space for only managingluminance image

Prescanning of image with GD (Gaussian Derivative) filters

Generating of edge map consisted of hard, medium, and softedges from the edge image

Applying GD filters to separated area in an edge map

Using a Gaussian smoothing filter for flat area Inversing YIQ to RGB area

AbstractAbstract


3/41

3 / 3

IntroductionIntroduction

Purpose

Sharpening the blurred image taken by a

conventional digital camera or scanner with normalsensor noise

Properties of classical sharpening models

Example : USM (unsharp masking)

Sensitivity to noise

Overshoot artifacts due to enhancing high-contrastarea

Inappropriateness for RGB

Insufficient denoising function


4/41

4 / 3

Diffusion model analogous to thermal image

Blurring

Thermal image Time varying image :

Blurred image :

First order approximation

Original sharp image

The same as Laplacian or simplest USM method

Review of Image sharpening modelsReview of Image sharpening models

ttfftf + )()0()(

)( 22222 yfxfkfktf +==

ggyfxfgf 22222 )( +

where : the diffusion constantk

(1)

(2)

(3)

.))(21()()0()( 222L+++= ttfttfftf

(4)


5/415 / 3

High-pass filter model and denosing operator

L-USM (linear USM)

Models to suppress noises in uniform area

A-USM (adaptive USM) model

Two directional Laplacian operators

Sensitivity for detail area with medium contrast

Insensitivity for uniform area

),(),(),(),(),(),( yxzyxyxzyxyxgyxf yyxx ++=),1(),1(),(2),( yxgyxgyxgyxzx +=

)1,()1,(),(2),( += yxgyxgyxgyxzy

where

),(),(),( yxZyxgyxf t+=

),(),(),( yxzyxgyxf +=

where : a positive scaling factor: a blurred input image

)1,()1,(),1(),1(),(4),( ++= yxgyxgyxgyxgyxgyxz

),( yxg

(5)

(6

(7

(8


6/416 / 3

Cubic USM (C-USM) model

Two types of operators

Sensitivity to high gradient edge

Less sensitivity to slow gradient edge

Separable cubic (SC-USM)

Nonseparable cubic (NSC-USM)

SNSC-USM

Average of the SC-USM and the NSC-USM

)]1,()1,(

),(2[)]1,()1,([)],1(

),1(),(2[)],1(),1([),(

2

2

+

+++

+=

yxgyxg

yxgyxgyxgyxg

yxgyxgyxgyxgyxz USMSC

)]1,()1,(),1(),1(),(4[

)]1,()1,(),1(),1([),( 2

+++

++=

yxgyxgyxgyxgyxg

yxgyxgyxgyxgyxz USMNSC

(9)

(1


7/417 / 3

Rational USM(R-USM) model

Higher enhancement in the detail zone

Lower enhancement in the uniform zone

Wavelet USM (W-USM) model

Multi-scale gradients of the wavelet transform

Independence of the variance of images and noises method in DIP book

FWT of the noise image with a wavelet function

Threshold detail coefficient with hard or soft thresholding Reconstruction with original approximation coefficient

)],(),(),(),([),(),( yxzyxcyxzyxcyxgyxf yyxx ++= (1

where ,

),(

),(),(

2

hyxkg

yxgyxc

x

xx

+

= ,),(

),(),(

2

hyxkg

yxgyxc

y

y

y

+

= (1

,)],1(),1([),( 2yxgyxgyxgx +=

.)]1,()1,([),( 2+= yxgyxgyxgx

(1


8/418 / 3

Lower-upper-middle (LUM) filter

Usage for both smoothing and sharpening

Insensitivity to additive noise and removing impulse noise


9/419 / 3

VisionVision--Based EdgeBased Edge--Sharpening OperatorSharpening Operator

Sharpening models

Gaussian derivative (=Hermite polynomialGaussian)

Gabor(=cosineGaussian)

DOG (Difference of Gaussian)

Difference-of-offset-Gaussian Difference-of-offset-(DOG)

Fig. 1. Typical edge-sharpening operat

based on the human visual field model.


10/4110 / 3

Basic equations of GD

Basic form of Gaussian

Second derivative

Edge signals and result of prescanning

.,2

exp2

1)( 222

2

2

2yxr

rrG +=

=

=

+=

2

2

2

2

2

22222

2exp1

2

1

)()()(

rr

yrGxrGrG

),(),(),( 2 yxgyxGyx =

(1

(1

(1


11/4111 / 3

MultiMulti--scale Adaptive Sharpening by Edgescale Adaptive Sharpening by Edge

ClassificationClassification

Fig. 2-1. Multi-scale edge adaptive imagesharpening process.

Procedure of multi-scale adaptive sharpening

Transform the RGB image to

YIQ image Different characteristics edges

depending on each channel

Preservation of gray balance onthe edge by using only luminance

part

Extract edge area byprescanning GD filter with

Design the edge histogram

Calculation H

S

321


12/4112 / 3

Fig. 2-2. Multi-scale edge adaptive image

sharpening process.

Segment into multiple edge

zones with hard, medium, soft

and flat gradients with

for making an edge mapfrom prescanned image

Apply GD filter with each

to sharpen the edge mapand Apply Gaussian

smoothing operator for flat

area (denoising)

Transform YIQ to RGB

H

,,( 321


13/4113 / 3

RGB

Edge image

edge histogram

Y

RGB

RGB to YIQ transform matrix

Prescanning with a GD filter andS

Calculation S

+=

),(),(

),(),(),(

yxgyxG

yxyxgyxf

m for edge area

for flat area

0),( yxM

0),( =yxM

edge map ),( yxM =),( yxM

),(3

),(2

),(1

),(00

4

42

21

1

yxfor

yxfor

yxfor

yxfor

S

S

S

S


14/41

14 / 3

GD filter design Conditions

The filter is approximated by an square matrix.

The weights of the GD filters should be equivalent to the

local integral of continuous GD functions in between

discrete lattice points.

The sum of GD filter weights is to be equal to zero so as

not to respond to flat signals.


15/41

15 / 3

Decision of a matrix for the first condition

Decision the size of Dependence on

Sufficient size to describe the receptive field :

Matrix of square

8M

][ ijw=W

Fig. 3. GD filter design in the zero sum condition.

zero cross point

minimum peak


16/41

16 / 3

Decision of weights for the second condition

Choice an odd integer Calculation weights between the lattice points

Compensation of weights for the third condition

Omission the negative weights outside of

Unsuitability for the zero sum condition

)12( += mM],[ ji

+

+

+=

5.0

5.0

5.0

5.0

2 )],([j

j

i

i

ij dxdyyxGw

8M

= =

+ >+==m

mi

m

mj

ij WWwW 0

where : the sum of the weights

: the positive entries of

: the positive entries of

W

, = =

++ =m

mi

m

mj

ijwW = =

=m

mi

m

mj

ijwW

+ijw

ijw

][ ijw

][ ijw

(2

(1


17/41

17 / 3

Modification the weights

New weight

Correction of only part

= =

=m

mi

m

mj

ijw 0'

where : corrected weights][][][ '' + += ijijij www

0=+ + kWW = ijij kww

'

= WWk /1

][ ijw

][][][ '' + +== ijijij www'

W

(21

(2

(2


18/41

18 / 3

Design of edge map

Decision of

Calculation of from prescanned image

Results from the experimental with the dozens of images Best choice : is around to

Ratio of :

)),(( yxM

=),( yxM

),(3

),(2

),(1

),(00

3

32

21

1

yxfor

yxfor

yxfor

yxfor

S

S

S

S


19/41

19 / 3

Application filters to an edge map

Transformation YIQ to RGB

+=

),(),(

),(),(),(

yxgyxG

yxyxgyxf

m for edge area

for flat area

0),( yxM

0),( =yxM

where is edge signals with

is a Gaussian smoothing filter

),( yxm

),( yxG

HHH aaa 3:2:),,( 321

=

B

G

R

Q

I

Y

311135.0522591.0211456.0

321263.0274453.0595716.0

114.0587.0299.0

(1


20/41

20 / 3

Experimental ResultsExperimental Results

Edge map tuning

Prescanning with

Example of histogramfrom prescanning image

Decision the range of edge map

Calculation of from prescanning image Choice of : around to

Usage of

Fig. 4. Typical dispersion in an edge

histogram dependent on image content

6.0=S

HHH aaa 3:2:),,( 321

1 H5.0 H8.0

H


21/41

21 / 3

Fig. 5. Map of standard deviation in an edge histogram for typical images.H

standard deviation of edge histogramH

Variety of standard deviations depending on images


22/41

22 / 3

Fig. 6(a). Edge map and edge histogram for image swallowtail with .5.35=H

HHH

9.0:6.0:3.0),,( 321

HHH

5.1:0.1:5.0),,( 321

HHH

4.2:6.1:8.0),,( 321

Test images to find optimal 1


23/41

23 / 3

HHH

9.0:6.0:3.0

),,( 321

HHH

5.1:0.1:5.0

),,( 321

HHH

4.2:6.1:8.0

),,( 321

Fig. 6(b). Edge map and edge histogram for image bride with .8.7=H


24/41

24 / 3

Comparison with improved USM methods for

a black and white image Test images

Lena1 : normal image with a small background

Lena2 : a blurred image with Gaussian noise

Lena3 : a degraded image corrupted by heavy impulse

noise

Value of parameters

for smoothing by Gaussian filter

6.0=S

8.1:2.1:6.0),,(321

8.0=f


25/41

25 / 3

Result

Better enhancement in the facial close-up with smoothedsurface and less noise

Fig. 7. Comparison with typical improved USM methods for black and white Lena1.


26/41

26 / 3


Better denoising in the facial area


27/41

27 / 3

Addition a well-known median filter as a preprocessor

MGD is not good reducing impulse noise

Better sharpened details in M-Russo

Better smoothed in facial area with proposed method


Sh F tSh F t


28/41

28 / 3

Sharpness FactorSharpness Factor

ES (edge sharpness)

Measurement the enhanced edge component

existing only in the edge areas

Calculation the integrated absolute amplitude ofedge enhancing signal divided by edge area

Obtainment

Counting the pixel numbers with in flat area ofedge map

E

E

A

dxdyyxsyxfES

=

),(),( filt

where is sharpening filter

is the amount of edge area

filts

EA

EA

0),( =yxM

(2


29/41

29 / 3

FS (frequency sharpness)

Enhancement Fourier spectra after sharpening

MSE (mean square error)

=

dVG

dVGFFS

)()(

)()()(

where is the original Fourier spectrais the sharpened Fourier spectra

is the human visual function

)(G)(F

)(V

= dxdyyxgyxfMSE2]),(),([

(2

(2


30/41

30 / 3

(noise power in flat area)

Noise power only in flat area

Result of experimental

fN

= dxdyyxgyxfyxENflatf

2

]),(),()[,(

where is a gate function to pass only flat area signals),( yxEflat

Table. 1. Comparison of image quality measure for Lena1.

(2


31/41

31 / 3



Sharpening for color imageSharpening for color image


32/41

32 / 3

Sharpening for color imageSharpening for color image

Procedure from a RGB image to YIQ image

Obtaining a gamma-corrected camera image

Transforming RGB into a linear sRGB andoperated on an RGB-to-YIQ linear matrix

Fig. 10. Edge coloring in RGB-independent sharpening versus Luminance Y sharpening.(a) Original (b) RGB independent (c) Sharpening Y in YIQ


33/41

33 / 3

(a) Original (b) Single GD (c) Proposed Multi GD (d) Edge map

(e) Edge histogram

(a) Original (b) Single GD (c) Proposed Multi GD (d) Edge map

(f) Edge histogram

Fig. 11. Tuned edge map and sharpened images.

Comparison of color sharpened images


34/41

34 / 3

(a) Original

(d) Original

(a) Original

(d) Original

(b) L-USM (c) Laplacian

(b) L-USM (c) Laplacian

(e) Single GD (f) Proposed Multi GD

(e) Single GD (f) Proposed Multi GDFig. 12. Comparison in sharpened color images.


35/41

35 / 3

(a) Original image A (b) Edge map (c) Edge histogram

(d) Original foreground

(g) Original background

(e) Single GD (f) Proposed Multi GD

(h) Single GD (i) Proposed Multi GD

Fig. 13-1. Adaptive sharpening effect with smoothing.


36/41

36 / 3

Fig. 13-2. Adaptive sharpening effect with smoothing.

(j) Original image B (k) Edge map

(l) Single GD (m) Proposed Multi GD

Edge Profile ComparisonEdge Profile Comparison


37/41

37 / 3

(a) Selected scan line in image (b) scan line on the edge map

(a) Enlarge image profiles before and after sharpening on scan line

Fig. 14. Adaptive sharpening effect with smoothing.

Edge Profile ComparisonEdge Profile Comparison

Sharpness Factors in Color ImagesSharpness Factors in Color Images


38/41

38 / 3

Sharpness Factors in Color ImagesSharpness Factors in Color Images

Fig. 15. Estimated quality factors.

Estimation

ES

Single GD > Multi-GD

Over-sharpening in single GD

FS Lift up the spatial frequency

components in the visible spatial

frequency range

Reducing noise

fN

Discussion and ConclusionDiscussion and Conclusion


39/41

39 / 3

A novel adaptive multi-filtering method

Work on the YIQ space

Usage of GD-filter with and considering

edge map, and Gaussian filter for flat area in edge

map to reduce noises Excellent results of sharpness and denoising

Drawback

Time-consuming

Discussion and ConclusionDiscussion and Conclusion

S H


40/41

40 / 3


41/41

41 / 3

Documents

20050430(Multiscale Image Sharpening Adaptive to Edge Profile)