Retinex AlgorithmCombined with Denoising Methods

Hae Jong, Seo Multi Dimensional Signal Processing Group

University of California at Santa Cruz

A

UCSC EE Dept Hae Jong

Background

New Approaches

Experimental Results

SSR, MSR, MSRCRSSR, MSR, MSRCR

Retinex Algorithm by two Bilateral filtersRetinex Algorithm by two Bilateral filters

Retinex Algorithm by two higher order Bilateral filtersRetinex Algorithm by two higher order Bilateral filters

Summary

Overview

UCSC EE Dept Hae Jong

Background on Retinex Algorithm

Part 1

UCSC EE Dept Hae Jong

Retinex AlgorithmRetinex Algorithm

Flow ChartFlow Chart

Kimmel et.al “A variational Framework for Retinex”

UCSC EE Dept Hae Jong

Retinex AlgorithmsRetinex Algorithms

Single Scale RetinexSingle Scale Retinex

Multi Scale RetinexMulti Scale Retinex

Multi Scale Retinex With Color restorationMulti Scale Retinex With Color restoration

Retinex Image Enhancement : Daniel J. Jonson et.al

Gaussian functionGaussian function

The weighted average version of different scale SSR

Different weight factor for different color bands

Given imageGiven imageReflectanceReflectance

UCSC EE Dept Hae Jong

Retinex AlgorithmRetinex Algorithm

Dynamic range compressionDynamic range compression

SharpeningSharpening

Color constancyColor constancy

Daniel J. Jonson et.al Retinex Image Enhancement :

UCSC EE Dept Hae Jong

Shortcoming?Shortcoming?

Amplify the noiseAmplify the noise

UCSC EE Dept Hae Jong

Retinex by Two Bilateral Filters

Part 2

Michael Elad “Retinex by Two Bilateral Filters”

UCSC EE Dept Hae Jong

Since the reflectance is passive, 0≤R≤1, we require S≤L and s≤ .

The illumination is supposed to be piecewise smooth.

Things to considerThings to consider

ds

s

22 Minimize

Trivial solution (L=255) should be avoided - The illumination should be forced to be close to s.

Michael Elad “Retinex by Two Bilateral Filters”

UCSC EE Dept Hae Jong

Noise is magnified in dark areas.

Forcing works againstnoise suppression.

smooth illumination envelope smooth reflectance

The Overall Model - shortcomingThe Overall Model - shortcoming

22222 Minimize

sss yxyx

sDDDD 22222

Minimize

sss yxyxs

DDDD

Requires an iterative solver!

Promotes hallows on the boundaries of the illumination.

rs

Michael Elad “Retinex by Two Bilateral Filters”

UCSC EE Dept Hae Jong

The bilateral filter is a weighted average smoothing, with weights inversely proportional to the radiometric distance and spatial distance between the center pixel and the neighbor [Tomasi and Manduchi, 1998]

The first Jacobi iteration that minimizes the above function leads to the bilateral filter [Elad, 2002]

zszz

B 2 Minimize

Bilateral FilterBilateral Filter

UCSC EE Dept Hae Jong

The Formulation with Bilateral Filter The Formulation with Bilateral Filter

rBsrBs rrs

22

,)( Minimize

With this new formulation:

Non-iterative solver can be deployed, Both the illumination and the reflectance are

forced to bepiece-wise smooth, thus preventing hallows,

Noise is treated appropriately.

Michael Elad “Retinex by Two Bilateral Filters”

Smooth illumination Smooth Reflectance

UCSC EE Dept Hae Jong

Numerical SolutionNumerical Solution

rBsrBs rrs

22

,)( Minimize

Part 1: Find by assuming r=0

Part 2: Given , find r by

Bilateral filter on s in an envelope mode

Bilateral filter on s- in a regular mode

Bsr

s

2 Minimize

rBsrrr

2)( Minimize

Part 1: Find by assuming r=0

Part 2: Given , find r by

Michael Elad “Retinex by Two Bilateral Filters”

illumination Reflectance

UCSC EE Dept Hae Jong

Higher order Bilateral filter on z- in a regular mode

New Suggestion – Higher order Bilateral New Suggestion – Higher order Bilateral

Part 1: Find by assuming r=0

i

Part 2: Given , find ri by i

Higher order Bilateral filter on z in an envelope mode

UCSC EE Dept Hae Jong

Returning Some IlluminationReturning Some Illumination

Kimmel et.al “A Variational Framework for Retinex”

UCSC EE Dept Hae Jong

Experiment Results

Part 3

Michael Elad “Retinex by Two Bilateral Filters”

UCSC EE Dept Hae Jong

15lP

Example 1Example 1

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

2000

Original Result (γ=3)

Parameter :100,3.0 lr

4rP 100,3.0 lr r Regular mode

Envelope mode

UCSC EE Dept Hae Jong

Example 2Example 2

Original Result (γ=3)

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

2000

15lPParameter :100,3.0 lr

4rP 100,3.0 lr r Regular mode

Envelope mode

UCSC EE Dept Hae Jong

Example 3Example 3

Original Result (γ=3)

500 1000 1500 2000 2500 3000 3500 4000

200

400

600

800

1000

1200

15lPParameter :100,3.0 lr

4rP 100,3.0 lr r Regular mode

Envelope mode

UCSC EE Dept Hae Jong

Example 4 ( Hallow Effect )Example 4 ( Hallow Effect )

50 100 150 200 250 300 350 400

50

100

150

200

250

300

100 200 300 400 500 600

50

100

150

200

250

300

350

400

450

Original Result (γ=3)

15lPParameter :100,3.0 lr

4rP 100,3.0 lr r Regular mode

Envelope mode

UCSC EE Dept Hae Jong

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

15lPParameter : 100,3.0 lr 4rP 100,3.0 lr

r

Bilater Filter VS Kernel Regression Bilater Filter VS Kernel Regression

Original Bilateral Filtered Result Kernel Regression Filtered Result

Regular modeEnvelope mode

UCSC EE Dept Hae Jong

Conclusion & Future workConclusion & Future work

Kimmel et.al “A Variational Framework for Retinex”

Implemented Retinex by two bilateral filters Implemented Retinex by two bilateral filters

It overcomes It overcomes hallowshallows, the need for , the need for iterationsiterations, and handles , and handles noise noise well. well.

Kernel regression method can do better Kernel regression method can do better using higher order. using higher order.

Apply Iterative Steering Kernel Regression Apply Iterative Steering Kernel Regression this frame work this frame work

UCSC EE Dept Hae Jong

Conclusion & Future workConclusion & Future work

Kimmel et.al “A Variational Framework for Retinex”

Implemented Retinex by two bilateral filters Implemented Retinex by two bilateral filters

It overcomes It overcomes hallowshallows, the need for , the need for iterationsiterations, and handles , and handles noise noise well. well.

Kernel regression method can do better Kernel regression method can do better using higher order. using higher order.

Apply Iterative Steering Kernel Regression Apply Iterative Steering Kernel Regression this frame work this frame work

UCSC EE Dept Hae Jong

Main References

[1] Elad.M, “Retinex by Two Bilateral Filters”, Scale-Space 2005, LNCS 3459, pp. 217-229, (2005).

[2] Rahman.Z, Jobson.D.J, Woodell.G.A : “Retinex processing for automatic image enhancement”. Journal of Electronic imaging, January (2004)

[3] Takeda.H, S.Farsiu, and P.Milanfar, “Kernel Regression for Image Processing and Reconstruction”, IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349-366, Feb. (2007) 2, 8

UCSC EE Dept Hae Jong

Thanks

Hae Jong, Seo

Email : rokaf@soe.ucsc.edu

Website : http://soe.ucsc.edu/~rokaf

UCSC EE Dept Hae Jong

Back up

Michael Elad “Retinex by Two Bilateral Filters”

UCSC EE Dept Hae Jong

s

Small Large

llumination as an Upper Envelope llumination as an Upper Envelope

ds Minimize 22

s

Michael Elad “Retinex by Two Bilateral Filters”

smooth illumination being close to s