1

A Markov Random Field Framework for Finding Shadows in a Single Colour Image

Cheng Lu and Mark S. DrewSchool of Computing Science, Simon Fraser University,

Vancouver (CANADA) {clu,mark}@cs.sfu.ca

2

Objective – finding shadows

Many computer vision algorithms, such as segmentation, tracking, and stereo registration, are confounded by shadows.

Finding shadows

3

Shadows stem from what illumination effects?

Changes of illuminant in both intensity and colour

Region Lit by Sky-light

only

Region Lit by Sunlight and

Sky-light

)/(},,{ BGRBGR

Intensity — sharp intensity

changes

Colour — shadows exist in the

chromaticity image

3/)( BGR

4

Colour of illuminantsWien’s approximation of Planckian illuminants:

How good is this approximation?

T

c

ecIE 2

51)(

2500 Kelvin

10000 Kelvin

5500 Kelvin

5

Invariant Image Concept

T

c

ii ecIE 2

51)(

kT

c

kkiii

kik

qeSIc

dQSEx

k

2

51

0

)()(

)()()()(

xnaFor narrow-band Sensors:

nai

Lambertian Surface

)()( kkk qQ

The responses:

Planckian Lighting

x

)(S

Finlayson et al.,ECCV2002

k = R, G, B

Shading and intensity term

6

Band-ratio chromaticity

G

R

B

Plane G=1

Perspective projection onto G=1

,2..1,/ kpkk

Let us define a set of 2D band-ratio chromaticities:

p is one of the channels,(Green, say) [or could use Geometric Mean]

7

Let’s take log’s:

Band-ratios remove shading and intensity

Teess pkpkkk /)()/log()log('

with ,)(51 kkkk qScs kk ce /2

Gives a straight line:

)(

)())/log(()/log(

1

21

'12

'2

p

ppp ee

eessss

Shading and intensity are gone.

8

Calibration: find illuminant direction

Log-ratio chromaticities for 6 surfaces under 14 different Planckian

illuminants, HP912 camera

Macbeth ColorChecker:

24 patches

Illuminant direction Invariant

direction

9

A real image containing shadows

The red line refers to the changes of illuminants: same surface lit by two

different lights

Two lights:

• Shadows : lit by sunlight and sky-light

• Non-shadows : lit by sky-light

10

Illuminant discontinuity

Illuminant discontinuity pair

Illuminant discontinuity pair:

• Two neighbouring pixels of a single surface, under two different lights

11

Illuminant discontinuity measure

ij

Using the means of two neighboring blocks of pixels • better than using two

neighbouring pixels because of noise and diffuse shadow edges.

Illuminant discontinuity angle:

• Cos of the two vectors||)||||(||

,

0

0

ij

ijijQ

0

12

Finding Shadows

First order neighbors 

Label image pixels with label l ={shadow, nonshadow}

Model this labelling problem using Markov Random Field

• The label of a pixel depends only on its neighbours

13

Markov Random Field

l is a Markov Random Field:

• l follows a Gibbs distribution: Z=normalizing constant, and

• U(l) is an energy function defined with respect to neighbours

labelling minimizing energy U(l)

)(1

1)(lU

TeZlP

14

Energy function

Dij=wQij+(1-w)RijCombining intensity difference Qij and illuminant discontinuity angle Rij (weight=w)

i Nj

jijiij

i

llllDlU )),(1(),()(

1),( ji ll if (li = lj)

0),( ji ll if ji ll Roughly,

In full,

15

Implementation

Gibbs Sampler can be used to minimize the energy: optimization technique.

Texture and noise may confuse the discontinuity measure, so the Mean Shift method is used to filter (segment) the image first.

16

Experiments