IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Assessment of interest point detectionalgorithms in OTB

Otmane Lahlou1, Julien Michel1, Damien Pichard1, JordiInglada2

1COMMUNICATIONS & SYSTÈMES2CENTRE NATIONAL D’ÉTUDES SPATIALES

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Introduction

Finding correspondences between images

I The dense approach: expensive but exhaustiveI The sparse approach: cheap, might be sufficient

Interest points

I Characteristic locations with highly discriminant keysI Robust: illumination, affine transform, noise . . .

In Orfeo ToolboxI Mainly SIFT and SURFI Perfect framework for a validation and comparison chain

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Outline of the presentation

Detectors in Orfeo Toolbox

Validation chain

Evaluation results

Scene classification

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Outline

Detectors in Orfeo Toolbox

Validation chain

Evaluation results

Scene classification

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Keypoints detectors in OTB

Scale Invariant Feature Transform (SIFT)

I Location: local extrema in scale space using DoG pyramidsI Key (128 values): local orientation histogramsI Implementation in OTB:

I Home-brewed version (not efficient)I Wrapping of SiftFast (very fast and accurate)

Speed-Up Robust Feature (SURF, variant of SIFT)

I Location: Laplacian approximation instead of DoGI Key (64 values): local Haar wavelet responseI Implementation in OTB: contributed by CS

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Example of application using OTB

Disparity map estimation based on sift matching

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Outline

Detectors in Orfeo Toolbox

Validation chain

Evaluation results

Scene classification

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Overall Scheme

InputImage

AffineWarping

Smoothing

Smoothing

Key pointdetection

Key pointdetection

Key pointMatching

Standard interface: Detector can be either SIFT or SURF

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Parameters

I Translation, rotation and scale factorI Image feature: band, intensity, NDVI, NDWII Amount of smoothing (anisotropic diffusion)I Number of scalesI Matching distanceI Back-matchingI Tolerance for match validation

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT Matching

134/269 good matches, 0 bad match

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SURF Matching

61/282 good matches, 1 bad match

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Outline

Detectors in Orfeo Toolbox

Validation chain

Evaluation results

Scene classification

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT scaling sensitivityAmplitude channel, rotation: 0, translation: (0,0), smoothing: no

0 50

100

150

200

250

300

0.4

0.6

0.8

1 1

.2 1

.4 1

.6

Quantity

Sca

ling

SIF

T 1

SIF

T 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SURF scaling sensitivitySURF is more sensitive to scaling than SIFT

0 50

100

150

200

250

300

350

400

450

500

0.4

0.6

0.8

1 1

.2 1

.4 1

.6

Quantity

Sca

ling

SU

RF

1S

UR

F 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT rotation sensitivityAmplitude channel, scale: 1., translation: (0,0), smoothing: no

0 50

100

150

200

250

300

-100

-80

-60

-40

-20

0 2

0 4

0 6

0 8

0 1

00

Quantity

Ang

le

SIF

T 1

SIF

T 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SURF rotation sensitivitySURF is highly sensitive to rotation (implementation ?)

0

100

200

300

400

500

600

-100

-80

-60

-40

-20

0 2

0 4

0 6

0 8

0 1

00

Quantity

Ang

le

SU

RF

1S

UR

F 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT smoothing sensitivityAmplitude channel, rotation: 5◦, translation: (5,3.3), scaling: 0.9

0 50

100

150

200

250

300

0 2

4 6

8 1

0 1

2 1

4

Quantity

Num

ber

of s

moo

thin

g ite

ratio

ns

SIF

T 1

SIF

T 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SURF smoothing sensitivitySURF is less sensitive to smoothing than SIFT

0 50

100

150

200

250

300

350

400

450

500

0 2

4 6

8 1

0 1

2 1

4

Quantity

Num

ber

of s

moo

thin

g ite

ratio

ns

SU

RF

1S

UR

F 2

Tru

e M

atch

esF

alse

Mat

ches

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT input type sensitivityrotation: 5◦, translation: (5,3.3), scaling: 0.9, smoothing: 5

0 50

100

150

200

250

300

Cha

nnel

1C

hann

el2

Cha

nnel

3C

hann

el4

Am

plitu

deN

dvi

Ndw

i

SIF

T 1

SIF

T 2

Goo

d m

atch

esB

ad m

atch

es

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SURF input type sensitivitySURF is less sensitive to the input type

0 50

100

150

200

250

300

350

400

450

500

Cha

nnel

1C

hann

el2

Cha

nnel

3C

hann

el4

Am

plitu

deN

dvi

Ndw

i

SU

RF

1S

UR

F 2

Goo

d m

atch

esB

ad m

atch

es

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

SIFT vs. SURF in OTB

pros consSIFT

I Fast implementation(SiftFast)

I Robust high ratematching

I Sensitive to input type

I Sensitive to smoothing

SURFI Robust wrt input types

I Robust wrt smoothing

I Poor matching rates

I Highly sensitive(implementation ?)

I Slower than SiftFast

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Outline

Detectors in Orfeo Toolbox

Validation chain

Evaluation results

Scene classification

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Scene classification (1)

Principles

I Keypoints spatial density: discriminant for classification ?I Data: BD Orfeo (patches of pan-sharpened Quickbird)

Examples of densities

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 0.01 0.02 0.03 0.04 0.05 0.06

His

togr

am

SIFT Density

SIFTDENSITYAMPLITUDE5A01

"histo_AMPLITUDE_5_A01.dat"

(a) Urban areas

0

0.05

0.1

0.15

0.2

0.25

0.3

0 0.01 0.02 0.03 0.04 0.05 0.06

His

togr

am

SIFT Density

SIFTDENSITYAMPLITUDE5C01

"histo_AMPLITUDE_5_C01.dat"

(b) Agricultural areas

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 0.01 0.02 0.03 0.04 0.05 0.06

His

togr

am

SIFT Density

SIFTDENSITYAMPLITUDE5D01

"histo_AMPLITUDE_5_D01.dat"

(c) Woods

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Scene Classification (2)

Decision RuleMaximum a posteriori

ResultsThis simple example shows promising results:

Urban Agricultural WoodsUrban 20 4 0

Agricultural 13 81 1Woods 3 3 114

IGARSS, July 12-17, 2009

Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion

Conclusion

Summary

I OTB is an efficient framework for algorithm validation chainI Behavior of SIFT/SURF wrt various parametersI Soundness of detectors for registration, but also

classification

Perspectives

I Exploit keys for object recognition tasks (work in progress)I Out-of core Sift/Surf extraction