Kartic Subr Cyril Soler Frédo Durand

Kartic Subr Cyril Soler Frédo Durand

Edge-preserving Multiscale Image Decomposition

based on Local Extrema

Edge-preserving Multiscale Image Decomposition

based on Local Extrema

INRIA, Grenoble Universities MIT CSAIL

Multiscale image decomposition

+

+

Medium

Pixels

Intensity

Input

Fine

Coarse

1D

Motivation

Detail enhancement

Separating fine texturefrom coarse shading

What is detail?

Some examples

Related work

Linear multiscale methods Edge-preserving approaches

1D Signal analysis

Related work: Linear multiscale methods

Edge-preserving approaches

1D Signal analysis

[Burt and Adelson 93]

[Rahman and Woodell 97]

[Pattanaik et al 98]

[Lindeberg 94]

Edges not preserved(Causes halos while editing)

Related work: Edge-preserving methods

1D Signal analysis

[Farbman et al 08] [Fattal et al 07]

[Bae et al 07] [Chen et al 07]

Edge-aware

Assume detail is low contrast

Related work: Empirical mode decomposition

Linear multiscale Edge-preserving approaches[Huang et al 98]

Developed for 1D signals

Detail depends on spatial scale

Not edge-aware

Input

Base layer

Detail layer(Input – Base)

+

Edge-preserving smoothing(e.g. bilateral filter)

Edge (preserved)

Detail (smoothed)

Existing edge-preserving image decompositions

Assume detail is low-intensity variation

Challenge: Smoothing high-contrast detail

Input

Challenge: Smoothing high-contrast detail

Edge

Low-contrast detail

High-contrast detail

Conservative smoothing (bilateral filter with narrow range-Gaussian)

Challenge: Smoothing high-contrast detail

Edge preserved?

Low-contrast detail smoothed?

High-contrast detail smoothed?

Challenge: Smoothing high-contrast detail

Edge preserved?

Low-contrast detail smoothed?

High-contrast detail smoothed?

Aggressive smoothing(bilateral filter with wide range-Gaussian)

Example: Smoothing high-contrast detail

Input [Farbman et al 2008] λ= 13, α = 0.2

[Farbman et al 2008] λ= 13, α = 1.2

Detail not smoothedDetail not smoothed

Coarse features smoothedEdge smoothed

Our approach: Use local extrema

Input

Local maxima

Local minima

Detail = oscillations between local extrema

Our approach: Use local extrema

Base = Local mean of neighboring extrema

Our approach: Use local extrema

Local mean of neighboring extrema

Edge preserved?

Low-contrast detail smoothed?

High-contrast detail smoothed?

Our detail extraction

Input

Base layer

Detail layer

+

High-contrastdetail smoothed

Edges preserved

Algorithm

Identify local extrema

Estimate smoothed mean

Detail at multiple scales

Input: Image + number of layers

Algorithm: Illustrative example

Algorithm: Identifying local extrema

Extrema detection kernel

Local maxima

Local minima

Algorithm: Estimating smoothed mean

1) Construct envelopes

Minimal envelope Interpolation preserves edge[Levin et al 04]

Maximal envelope

Algorithm: Estimating smoothed mean

2) Average envelopes

Estimated mean

Algorithm: After one iteration

+

Input

Base

Detail

Algorithm: Mean at coarser scale

Local maxima

Local minima

Widen extrema detection kernel

Algorithm: Mean at coarser scale

Minimal envelope

Maximal envelope

Algorithm: Mean at coarser scale

Estimated mean

Identify local extrema

Construct envelopes

Average envelopes

Recap: Detail extraction

Smoothed mean

Detail = Input - BaseBase

Input

Base B2

Base B1

Input

Detail D2

Detail D1

Recap: Multiscale decomposition

Layer 1Layer 2Layer 3 Iteration 1on input

Iteration 2on B1

Recurse n-1 times for n-layers

Coarse Fine

Results

Results: Smoothing

Input

Smoothed

Results: Multiscale decomposition

Medium

Input

FineCoarse

Low contrast edge High contrast detailLow contrast edge High contrast detail

Results: Multiscale decomposition

Input

Results: Multiscale decomposition

FineCoarse

Applications: Image equalization

Applications: Smoothing hatched images

Applications: Coarse illumination transfer

Applications: Coarse illumination transfer

Applications: Coarse illumination transfer

Applications: Tone-mapping HDR images

Comparison

[Farbman et al 2008]Our Result

Our smoothing

Limitation

Input Our Result

Conclusion

Detail based on local extrema

Smoothing high contrast detail

Edge-preserving multiscale decomposition

Acknowledgements

INRIA post-doctoral fellowship

Equipe Associée with MIT ‘Flexible Rendering’

Adrien Bousseau & Alexandrina Orzan

HFIBMR grant (ANR-07-BLAN-0331)

Anonymous reviewers

C++ source: http://artis.imag.fr/~Kartic.Subr/research.html