1

Manifold Alignment for Multitemporal Hyperspectral Image Classification

H. Lexie Yang1, Melba M. Crawford2

School of Civil Engineering, Purdue Universityand

Laboratory for Applications of Remote Sensing

Email: {hhyang1, mcrawford2}@purdue.eduJuly 29, 2011

IEEE International Geoscience and Remote Sensing Symposium

2

Outline

• Introduction• Research Motivation

− Effective exploitation of information for multitemporal classification in nonstationary environments

− Goal: Learn “representative” data manifold• Proposed Approach

− Manifold alignment via given features− Manifold alignment via correspondences− Manifold alignment with spectral and spatial information

• Experimental Results• Summary and Future Directions

3

Introduction

• Challenges for classification of hyperspectral data− temporally nonstationary spectra− high dimensionality

2001 2003 2004 2005 200620022001

June July May May May May June

N na

rrow sp

ectra

l ban

ds

123

N>>30

4

• Nonstationarities in sequence of images − Spectra of same class may evolve or drift

over time

• Potential approaches− Semi-supervised methods− Adaptive schemes− Exploit similar data geometries

Explore data manifolds

Research Motivation

Good initial conditions required

5

Manifold Learning for Hyperspectral Data

• Characterize data geometry with manifold learning − To capture nonlinear structures − To recover intrinsic space (preserve spectral neighbors) − To reduce data dimensionality

• Classification performed in low dimensional space

Spectr

al ba

nds

Spatial dimension

Spa

tial d

imen

sion

1234

2nd dim 1st dim

3rd dim

n

56

Original space Manifold space

6

Challenges: Modeling Multitemporal Data

• Unfaithful joint manifold due to spectra shift

• Often difficult to model the inter-image correspondences

Data manifold at T1 Data manifold at T2 Data manifolds at T1 and T2

7

Proposed Approach: Exploit Local Structure

Assumption: local geometric structures are similar Approach: Extract and optimally align local geometry

to minimize overall differences

Locality

Spectral space at T1 Spectral space at T2

8

Proposed Approach: Conceptual Idea

(Ham, 2005)

9

Proposed Approach: Manifold Alignment

• Exploit labeled data for classification of multitemporal data sets

Samples with no class labels

Joint manifold

Samples with class labels

10

Manifold Alignment: Introduction

• and are 2 multitemporal hyperspectral images− Predict labels of using labeled

• Explore local geometries using graph Laplacian and some form of prior information

• Define Graph Laplacian

− Two potential forms of prior information: given features and pairwise correspondences [Ham et al. 2005]

1I 2I

L

where0 , otherwise 1 , neighbors of ij

i j

ii ijj

L D W

Wx x

D W

2I 1I

11

Manifold Alignment via Given Features

Given Features is Joint Manifold *F

Minimize ( )C F

1

1 2

{ ,..., }: Given features of labeled samples : Graph Laplacian of and : Relative weighting coefficient

i ns s s nL I I

1 1 1 21 2

2 1 2 2

, ,,

, ,

I I I II I

I I I I

L LL

L L

n

i

IITiiFF

FLFsfFCF 21 ,* minarg)(minarg

12

Manifold Alignment via Pairwise Correspondences

Correspondences between and 1I 2I

Minimize ( , )C F G1IL

2IL

Joint Manifold * *[ ; ]F G

1

2

1 1 1 2

1 2

1

{ ,..., } ;{ ,..., }( , ) : Pairwise correspondences in [ ; ] where index corresponds to pair ( , ) extracted from and

: Graph Laplacian of

: Graph Laplacian

N M

i i

i i

I

I

x x I y y If g F G

i x y I IL I

L

2of

: Relative weighting coefficent

I

k

iI

TI

TiiGFGF

GLGFLFgfGFCGF21,,

** minarg),(minarg];[

13

MA with spectral and spatial information

• Combine spatial locations with spectral signatures− To improve local geometries (spectral) quality− Idea: Increase similarity measure when two samples are

close togetherWeight matrix for graph Laplacian:

where spatial location of each pixel is represented as

2 2spa spe

Spatial Distance ( , ) Spectral Distance ( , )exp expi j i j

ij

z z x xW a

ix2Riz

14

Experimental Results: Data Three Hyperion images collected in

May, June and July 2001 May - June pair: Adjacent

geographical area June - July pair: Targeted the same

area

May June July

Class

Water

Floodplain

Riparian

Firescar

Island interior

Woodlands

Savanna

Short mopane

Exposed soils

15

Experimental Results: Framework

Joint manifoldGraph

LaplacianPrior information

Given features

Correspondences

Develop Data Manifold of

Pooled Data

GF

CF

PF

Data sets Labels

Pair 1 Pair 2

May June Training data For KNN classifier

June July Testing data For overall accuracy evaluation

Classificationwith KNN

I1, I2L

I1L I2L

16

Manifold Learning for Feature Extraction

• Global methods consider geodesic distance − Isometric feature mapping

(ISOMAP)

• Local methods consider pairwise Euclidian distance− Locally Linear Embedding (LLE): (Saul and Roweis, 2000)− Local Tangent Space Alignment (LTSA): (Zhang and Zha,

2004)− Laplacian Eigenmaps (LE): (Belkin and Niyogi, 2004)

(Tenenbaum, 2000)

17

MA with Given Features

ISOMAP

LTSA

LLE

LE

75.69

70.18

47.65

62.38

0.620000000000006

7.69999999999999

29.64

16.63

Pooled DataAccuracy increment (Δ) with MA using extracted features

Overall Accuracy

• Baseline: Joint manifold developed by pooled data

(May, June pair)

79.21

77.29

77.88

76.31

18

MA Results – Classification Accuracy

• Evaluate results by overall accuracies

MethodsOverall Accuracy

May, June June, July

Manifold learning from pooled data 62.38% 83.00%

Manifold alignment(MA)

Given features (LE) 79.21% 86.16%Correspondences 81.22% 84.27%

MethodsOverall Accuracy

May , June June, July

Given features (LE)

Spectral 79.21% 86.16%Spectral + spatial 84.21% 90.30%

CorrespondencesSpectral 81.22% 84.27%Spectral + spatial 84.74% 90.11%

19

Results – Class Accuracy

May/June Pair

Typical class(Island Interior) Critical class

(Woodlands)Critical class

(Riparian)

MA: Given Features: SpectralMA: Correspondences: Spectral

MA: Given Features: Spectral / SpatialMA: Correspondences: Spectral / Spatial

Pooled data

20

Summary and Future Directions

• Multitemporal spectral changes result in failure to provide a faithful data manifold

• Manifold alignment framework demonstrates potential for nonstationary environment by utilizing similar local geometries and prior information

• Spatial proximity contributes to stabilization of local geometries for manifold alignment approaches

• Future directions− Investigate alternative spatial and spectral integration

strategy− Address issue of longer sequences of images

21

Thank you.Questions?

22

References

• J. Ham, D. D. Lee, and L. K. Saul, “Semisupervised alignment of manifolds,” in International Workshop on Artificial Intelligence and Statistics, August 2005.

23

Backup Slides