Extreme Value Theory and Dimension reduction for the study of hyperspectral images

Extreme ValueTheory andDimension

reduction for thestudy of

hyperspectralimages

LaurentGARDES

Inference onWeibull taildistributions

Extremeconditionalquantileestimation

Dimensionreduction andregression

Further works

1

Extreme Value Theory and Dimensionreduction for the study of hyperspectral images

Laurent GARDES

INRIA Rhone-Alpes, LJK, Team MISTIShttp://mistis.inrialpes.fr/people/gardes/

Habilitation a diriger des recherches



hyperspectralimages

LaurentGARDES




Further works

2

Outline

I. Inference on Weibull tail distributions.

II. Extreme conditional quantile estimation.

III. Dimension reduction and regression.

IV. Further works.



hyperspectralimages

LaurentGARDES




Further works

3

1 Inference on Weibull tail distributions

2 Extreme conditional quantile estimation

3 Dimension reduction and regression

4 Further works



hyperspectralimages

LaurentGARDES




Further works

4

In collaboration with

J. Diebolt (D.R. CNRS)

S. Girard (C.R. INRIA)

A. Guillou (Professeur, Universite de Strasbourg)

Associated publications

TEST (2008)

JSPI (two in 2008)

REVStat (2006)

CIS (2005)



hyperspectralimages

LaurentGARDES




Further works

5

Recalls on Extreme Value Theory

Let X1, . . . ,Xn be n independent random variables with the samecumulative distribution function F . The order statistics are denotedby X1,n ≤ . . . ≤ Xn,n.

Goal: Extreme quantile estimation i.e. for αn → 0 as n→∞,estimation of

q(αn) = F←(αn).

Main difficulty: If αn is small (nαn → 0),

P(q(αn) > Xn,n)→ 1.

An extrapolation is thus needed!



hyperspectralimages

LaurentGARDES




Further works

6




hyperspectralimages

LaurentGARDES




Further works

7


The main result is the extreme value theorem:

Theorem If there exist two sequences (an > 0), (bn) and γ ∈ R suchthat

P

Xn,n − bn

an≤ x

ff→ Hγ(x),

then

Hγ(x) =

exp[−(1 + γx)

−1/γ+ ] if γ 6= 0,

exp(−e−x ) if γ = 0,

• Hγ(.) is the extreme value distribution.

• γ is the extreme value index.



hyperspectralimages

LaurentGARDES




Further works

8


Three maximum domains of attraction (MDA)

• γ > 0: Frechet MDA (Pareto, student, Cauchy, . . .)

• γ < 0: Weibull MDA (uniform)

• γ = 0: Gumbel MDA (normal, Weibull, exponential, log-normal,etc . . .)



hyperspectralimages

LaurentGARDES




Further works

9

Weibull tail distributions

• Sub family of the Gumbel MDA.• The survival function is given by:

F (x) = expn−x1/θL(x)

o, θ > 0.

L(.) is a slowly varying function: for all λ > 0,

limx→∞

L(λx)

L(x)= 1.

• θ is the Weibull tail-coefficient.

• Examples of Weibull tail distributions: Weibull, normal, gamma,exponential, etc . . .

• Log-normal is not a Weibull tail distribution.



hyperspectralimages

LaurentGARDES




Further works

10

Weibull tail distributions

• An estimator of θ was proposed by Beirlant et al. (1996)

θBn =

kn−1Xi=1

(log(Xn−i+1,n)− log(Xn−kn+1,n))

,kn−1Xi=1

(log2(n/i)− log2(n/kn)) ,

where log2(.) = log log(.) and (kn) is a sequence of integers suchthat 1 < kn < n.

• The corresponding extreme quantile estimator is defined by:

qB(αn) = Xn−kn+1,n

„log(1/αn)

log(n/kn)

«θBn

.

• The asymptotic properties of these estimators are notestablished by the authors.



hyperspectralimages

LaurentGARDES




Further works

11

Contributions

• Generalizations of the estimator θBn .

− Introducing weighted estimators.

kn−1Xi=1

W (i/kn)(log(Xn−i+1,n)−log(Xn−kn+1,n))

,kn−1Xi=1

W (i/kn)(log2(n/i) − log2(n/kn))

− Using others normalizing sequences.

1

Tn

kn−1Xi=1

(log(Xn−i+1,n)− log(Xn−kn+1,n)), Tn ∼kn

log(n/kn).

− Bias corrected estimator of θ.

• Generalization of qB(αn) by replacing θBn by any other estimator

of θ.

• Bias corrected estimator of q(αn).



hyperspectralimages

LaurentGARDES




Further works

12

Bias corrected estimator of θ

The estimators are based on the following approximation: for α andβ small enough:

q(α)

q(β)=

„− log(α)

− log(β)

«θ `(− log(α))

`(− log(β))≈„− log(α)

− log(β)

«θ.

A second order condition is required in order to specify the bias term:

(H.1) There exist ρ < 0 and a function b(.) satisfying b(x)→ 0 asx →∞ such that locally uniformaly,

limx→∞

log(`(λx)/`(x))

b(x)Kρ(λ)= 1,

where Kρ(λ) = (λρ − 1)/ρ.

The function b(.) ∈ RVρ i.e. b(x) = xρ`∗(x) where `∗(.) is a slowlyvarying function.



hyperspectralimages

LaurentGARDES




Further works

13


Under (H.1), we have approximately:

Zj ≈ θ + b(log(n/kn))xj + ηj , j = 1, . . . , kn,

where Zj = j log(n/j)(log(Xn−j+1,n)− log(Xn−j,n)),xj = log(n/kn)/ log(n/j) and ηj is an error term.

Ignoring the bias term, leads to the estimator k−1n

∑kn

j=1 Zj .

Estimating θ and b(log(n/kn)) by the method of least-squares leadsto the bias corrected estimator:

θDn =

1

kn

knXj=1

Zj −b(log(n/kn))

kn

knXj=1

xj ,



hyperspectralimages

LaurentGARDES




Further works

14


Theorem Under (H.1), if x |b(x)| → ∞ as x →∞ and

k1/2n

log(n/kn)b(log(n/kn))→ Λ 6= 0,

then,k

1/2n

log(n/kn)(θD

n − θ)d→ N (0, θ2).

• Condition x |b(x)| → ∞ implies that ρ ≥ −1.

• θDn converges to θ with the same rate of convergence as θB

n butwithout asymptotic bias.



hyperspectralimages

LaurentGARDES




Further works

15


Illustration with a simulation of N = 100 samples of size n = 500from a N (0, 1) distribution (θ = 1/2).

Horizontal axis: kn. Vertical axis: mean of the estimator (left) andMSE of the estimator (right). In black: θB

n and in grey: θDn .



hyperspectralimages

LaurentGARDES




Further works

16

Further works

• Propose a new model that incompasses Weibull tail and Heavytail distributions (work accepted in JSPI).

• Use the previous model to construct statistical hypothesis teston the tail distribution.

• Prove asymptotic results on the estimators when theobservations are not independent.

• Estimation of the second order parameter ρ.



hyperspectralimages

LaurentGARDES




Further works

17




4 Further works



hyperspectralimages

LaurentGARDES




Further works

18


A. Daouia (MCF, Universite Toulouse 1)


Supervision research activities

E. Ursu (Postdoctoral fellow)

A. Lekina (past PhD student)

J. el-Methni (new PhD student)

Contracts and grants

ANR (French Research Agency), program VMC

CEA (Atomic Energies Commission) of Cadarache


TEST (to appear)

Extremes (2010)

JMVA (2008 and 2010)



hyperspectralimages

LaurentGARDES




Further works

19

Motivation

Rainfalls data in the Cevennes-Vivarias region

Data provided by Laboratoire d’etude des Transferts en Hydrologie etEnvironnement (LTHE). Work supported by the ANR.

Horizontal axis: longitude. Vertical axis: latitude. Color scale: altitude.



hyperspectralimages

LaurentGARDES




Further works

20

Motivation

• 142 rain gauge stations.

• Hourly rainfalls measured from 1993 to 2000.

• Total number of observations: n = 264056.

• Y is the variable of interest (hourly rainfall).

• X is the covariate recorded with Y (longitude + latitude +altitude).

• Aim: estimate the amount of rain expected to be exceeded onceevery T years i.e estimate the T -year return level.



hyperspectralimages

LaurentGARDES




Further works

21

Framework

• The conditional survival distribution function of Y when thecovariate is equal to x is:

F (y , x) = y−1/γ(x)L(y , x).

• γ(.) is an unknown positive function called the conditional tailindex.

• For x fixed, L(., x) is a slowly varying function.

• In early studies, the conditional distribution is assumed to beGumbel: unrealistic assumption.



hyperspectralimages

LaurentGARDES




Further works

22

Main contributions

Two situations were investigated:

• Random covariate

− Estimation of the conditional tail index γ(x).− Estimation of conditional extreme quantiles.− Estimation of small tail probabilities.

• Deterministic covariate

− Estimation of the conditional tail index γ(x).− Estimation of conditional extreme quantiles.− Application to the estimation of return levels.



hyperspectralimages

LaurentGARDES




Further works

23

Methodology

• Let (Y1, x1), . . . , (Yn, xn) be a sample of independentobservations

• To estimate the conditional quantile q(αn, x), only theobservations for which the associated covariates are ”close” to xare required.

• We consider the observations Yi for which the associatedcovariates belong to the ball B(x , hn,x) (ball centered at point xwith radius hn,x → 0 as n→∞).



hyperspectralimages

LaurentGARDES




Further works

24

Methodology

• These observations are denoted by Z1, . . . ,Zmn,x . The number ofcovariates in the ball B(x , hn,x) is denoted by mn,x .

• Corresponding order statistics: Z1,mn,x ≤ . . . ≤ Zmn,x ,mn,x .



hyperspectralimages

LaurentGARDES




Further works

25

Estimation of q(αn, x)

We propose the following estimator:

q(αn, x) = Zmn,x−kn,x +1,mn,x

„kn,x

αnmn,x

«γn(x)

.

• kn,x is a positive sequence such that 1 < kn,x < mn,x .

• Estimator in the same spirit as the Weissman (1978) estimator.

• γn(x) estimator of γ(x) such that:

k1/2n,x (γn(x)− γ(x))

d→ N (0,AV(x)).



hyperspectralimages

LaurentGARDES




Further works

26

Estimation of q(αn, x)

Two sources of bias

• The bias introduced by the slowly varying function. It iscontrolled via the function b(., x) ∈ RVρ(x) (that appears in asecond order condition).• The bias due to the conditional framework. It is controlled by

ωn(a) = sup

˛log

q(α, t)

q(α, t′)

˛, α ∈]a, 1− a[, (t, t′) ∈ B(x , hn,x )2

ff.



hyperspectralimages

LaurentGARDES




Further works

27

Asymptotic normality

Under a second order condition, if kn,x →∞, mn,x/kn,x →∞, ifthere exists δ2 > 0 such that

k2n,xωn(m

−(1+δ2)n,x )→ 0 and k

1/2n,x b(mn,x/kn,x , x)→ 0,

then, if αn is such that αn → 0 and bmn,xαnc → c ∈ N,

k1/2n,x

log(kn,x/(mn,xαmn,x ))

„q(αn, x)

q(αn, x)− 1

«d→ N (0,AV(x)).

• The asymptotic normality is provided by γn(x).



hyperspectralimages

LaurentGARDES




Further works

28

Return levels estimation

• Estimation in the whole region of the 10-years return level.

• Conditional extreme quantile of order 1/(365× 24× 10).

• ”Extreme” since there are only 7 years of observations.

Montpellier

Nimes

Ales

Privas

ValenceLe Puy

Mende

Millau

75

80

85

90

95

100

105

110

115

• Return level is globally decreasing with the altitude.

• Altitude is not the unique factor.

• Result validated by the LTHE.



hyperspectralimages

LaurentGARDES




Further works

29

Further works

• Estimation of frontier functions (work accepted in the book”Festschrift in honor of L. Simar, Springer”)

• Estimation of conditional extreme quantile under more generalassumptions on the tail distribution.

• Choice of the sequences (hn,t) and (kn,t).

• In a short simulation study, we remark that:

− temporal dependence does not affect the bias (but increases thevariance).

− spatial dependence slightly affects the bias.

Make a theoretical study to confirm these points.



hyperspectralimages

LaurentGARDES




Further works

30




4 Further works



hyperspectralimages

LaurentGARDES




Further works

31


S. Doute (C.R. CNRS, LPG)


Supervision research activities

C. Bernard-Michel (postdoctoral fellow)

M. Fauvel (postdoctoral fellow)

Contracts and grants

ANR MDC0


Journal of Geophysical Research (2009)

Statistics and Computing (2009)

Biometrics (2008)



hyperspectralimages

LaurentGARDES




Further works

32

Motivation• Study of hyperspectral images from south polar region of Mars

(work supported by the ANR, in collaboration with theLaboratoire de Planetologie de Grenoble (LPG)).• Spectra (of dimension d ≈ 184) collected by the imaging

spectrometer OMEGA aboard MARS express.• Aim: estimate some ground properties (proportion of water,

CO2, etc . . .)



hyperspectralimages

LaurentGARDES




Further works

33

Motivation

• Learning data-set (Xi ,Yi ), i = 1, . . . , n (n ≈ 30000)

• Yi ∈ R: given value of a physical parameter (for exampleproportion of water).

• Xi ∈ Rd (d = 184): the associated spectra obtained by theradiative transfert algorithm.• Aim: estimate the link function G defined by:

Yi = G(Xi , ηi ), i = 1, . . . , n,

where ηi is a random error.

• Estimation of G is difficult (curse of dimensionality) when d islarge.

• Existing methods:

− Nearest neighbors.− Partial Least Square regression (PLS).− Support Vector Machine regression (SVM).− Sliced Inverse Regression.



hyperspectralimages

LaurentGARDES




Further works

34

Recall on SIR method

• Single index model: (Xi ,Yi ) independent observations from therandom vector (X ,Y ) ∈ Rd × R satisfying

Y = g(βtX , η), η and X independent.

• All the information on Y contained in X is also contained inβtX .

• β ∈ Rd is the effective dimension reduction (edr) direction.

• Estimation of G : Rd → R replaced by the estimation ofg : R→ R

• Estimation of β:

• divide the support of Y into H slices S1, . . . , SH .• maximize the estimated variance between slices.



hyperspectralimages

LaurentGARDES




Further works

35

Illustration of SIR method



hyperspectralimages

LaurentGARDES




Further works

36


• The estimator β of the e.d.r. direction is solution of theproblem:

arg maxβ

βt Γβ under the constraint βt Σβ = 1.

where

Σ =1

n

nXi=1

(Xi − X )(Xi − X )t , with X =1

n

nXi=1

Xi ,

and

Γ =1

n

HXj=1

nj (Xj − X )(Xj − X )t , with Xj =1

nj

Xi :Yi∈Sj

Xi .

• β is the eigenvector of the matrix Σ−1Γ associated to the largesteigenvalue.



hyperspectralimages

LaurentGARDES




Further works

37


Other way to introduce the SIR estimator β

• Inverse regression model (proposed by Cook (2007)):

X = µ+ Vβ

H−1Xj=1

cj sj (Y ) + ζ,

where µ ∈ Rd , V is a d × d covariance matrix, ζ a Nd(0,V )random vector independent of Y , cj ∈ R and sj : R→ R, forj = 1, . . . ,H − 1.• Taking

sj (.) = I. ∈ Sj −nj

n, where nj =

nXi=1

IYi ∈ Sj, j = 1, . . . ,H − 1,

the maximum likelihood estimator of β is (up to a scaleparameter) the SIR estimator β.



hyperspectralimages

LaurentGARDES




Further works

38

Contribution

SIR requires the inversion of Σ. When d is large, Σ is ill-conditioned.

• Proposition of a regularized SIR version in order to avoid theinversion of Σ.

• Application to hyperspectral images from Mars.



hyperspectralimages

LaurentGARDES




Further works

39

Regularized SIR method

• Introduction of a Gaussian prior in Cook’s model: conditionallyto (ρ,Y ),

Θ =

0@ρ−1/2H−1Xj=1

cj sj (Y )

1Aβ ∼ Nd (0,Ω),

where ρ = (βtΣβ)/(βtVβ).

• The prior covariance matrix Ω is known.

• conditionally to (ρ,Y ), Θ is proportional to β.

• Ω describes which directions are most likely to contain β.• The ”MAP” estimator of β is the eigenvector associated to the

largest eigenvalue of:(ΩΣ + Id )−1ΩΓ,

• The inversion of Σ is not required.



hyperspectralimages

LaurentGARDES




Further works

40

Regularized SIR method

Links with existing methods

• Taking Ω = Σ−1 leads to SIR. Directions corresponding to smallvariance are most likely.

• Regularized SIR also incompasses the ”Ridge-regularization” andthe ”PCA+SIR” regularization.

New method

• Taking Ω = τ−1Σ leads to ”Tikhonov-SIR”. τ is a regularizationparameter. Directions corresponding to large variance are mostlikely.



hyperspectralimages

LaurentGARDES




Further works

41

Application to hyperspectralimages from Mars

Estimation of the proportion of CO2 by SIR (left) and RegularizedSIR (right).



hyperspectralimages

LaurentGARDES




Further works

42

Further works

• Asymptotic results for the regularized SIR estimator of β (jointwork with A.F. Yao, Universite Aix-Marseille II)

• Add a spatial regularization.

• Propose regularization methods for other regression tools (PLS,Nearest Neighbors, etc . . .).



hyperspectralimages

LaurentGARDES




Further works

43




4 Further works



hyperspectralimages

LaurentGARDES




Further works

44

Further works

• Study of conditional Weibull tail distributions (”Weibull tail” +”conditional extremes”).

• Dimension reduction for the estimation of conditional extremequantiles (”conditional extremes” + ”Dimension reduction”).

• Use other statistical tools for the study of extreme values (forexample, use nonparametric estimation methods to estimate theslowly varying function).



hyperspectralimages

LaurentGARDES




Further works

45

Summary (Period 2004-2010)

• 13 articles in international scientific journals.

• 2 book chapters.

• 2 PhD students.

• 3 postdoctoral fellows.

• 1 contract with the CEA.

• 2 financial supports by the ANR.

Documents

Extreme Value Theory and Dimension reduction for the study of hyperspectral images