Stat 598 Presentation on Nonseperable, Stationary ...chen418/study_research/stat598... · Stat 598 Presentation on Nonseperable, Stationary Covariance Functions for Space-Time Data

Stat 598 Presentation onNonseperable, Stationary Covariance Functions for

Space-Time Data

Xiongzhi Chen

Department of Statistics

November 16, 2010

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 1 / 22

Outline

Positive Definite Functions

Main Mathematical Results

Application to Irish Wind Data

Why Such A Correlation Model

Other Strategies and Wrong Claims

Proofs

Acknowledgements


Key Component of Spatial-Temporal Modelling

Observations generated from a square-integrable stochastic processZ =

{Z (s; t), (s; t) ∈ Rd ×R

}Process Z induces a family of functionsC =

{Cs ,t : Rd ×R→ R| (s; t) ∈ Rd ×R

}, where

Cs ,t (h; u) = Cov ((Z (s; t) ,Z (s + h; t + u))

at each (s; t).

Covariance stationarity is equivalent to

C = {C (h; u)}

Then C (·; ·) is called the covariance function of Z.


Positive-Definite (PSD) Functions

A function f : Rd+1 → K is positive definite iff

k

∑i ,j=1

f (wi − wj ) aia∗j ≥ 0

First equivalence

f being a covariance function ⇔ f positive definite

Second equivalence (Bochner’s Theorem): A continuous functionf on Rd+1 is positive definite iff

f (w) =∫exp

(√−1 〈η,w〉

)dF (η) (1)

Equivalences and Spectral Representation

Proof of Bochner’s Theorem


Construction of PSD Functions

Main method: via Laplace transform and Bochner’s theorem.

Cressie and Huang (1999) [1] based on closed-form F-inverse of

Cω (u) =∫e−ih

′ωC (h; u) dh (2)

where C (h; u) ∈ Cb ∩ L1 and

C and Cω both are covariance functions or neither are (3)

Obviously C ≡ Cω mod (F) but C might be un-usable.

Gneiting (2002) [2] (i) avoids explicit F-inverse of Cω and usesL-representation of completely monotone (cm) functions andthe Bochner’s theorem; (ii) produces explicit product form ofsome PSD functions

Completely Monotone Functions


Construction of PSD Functions

Main method: via Laplace transform and Bochner’s theorem.

Cressie and Huang (1999) [1] based on closed-form F-inverse of

Cω (u) =∫e−ih

′ωC (h; u) dh (2)

where C (h; u) ∈ Cb ∩ L1 and

C and Cω both are covariance functions or neither are (3)

Obviously C ≡ Cω mod (F) but C might be un-usable.

Gneiting (2002) [2] (i) avoids explicit F-inverse of Cω and usesL-representation of completely monotone (cm) functions andthe Bochner’s theorem; (ii) produces explicit product form ofsome PSD functions

Completely Monotone Functions


Key Mathematical Result

Theorem 2 (Gneiting (2002) [2]): Let k, l ∈N+, σ2 > 0.Let ϕ (t) , t ≥ 0be a cm function, and let ψ (t) , t ≥ 0, be a positive function with a cmderivative. Then

C (h; u) =σ2

ψ(‖u‖2

)k/2 ϕ

‖h‖2

ψ(‖u‖2

) , (h; u) ∈ Rk ×Rl (4)

is a space-time covariance function.

The family includes almost all examples given in Cressie and Huang(1999) [1]

In application, ϕ,ψ can be associated with the data’s spatial andtemporal structures, respectively

Note how the temporal correlation is "scaled" by spatialcorrelation




C (h; u) =σ2

ψ(‖u‖2

)k/2 ϕ

‖h‖2

ψ(‖u‖2

) , (h; u) ∈ Rk ×Rl (4)








C (h; u) =σ2

ψ(‖u‖2

)k/2 ϕ

‖h‖2

ψ(‖u‖2

) , (h; u) ∈ Rk ×Rl (4)






Examples of Building Functions

ϕ1 (t) = exp (−ctγ) c > 0,γ ∈ (0, 1]ϕ2 (t) =

(2ν−1Γ (ν)

)−1 (ct1/2)νKν

(ct1/2) c > 0, ν > 0

ϕ3 (t) = (1+ ctγ)−ν c > 0, ν > 0,γ ∈ (0, 1]

ϕ4 (t) = 2ν((exp

(ct1/2))+ exp (−ct1/2))−ν

c > 0, ν > 0ψ1 (t) = (at

α + 1)β a > 0, α ∈ (0, 1], β ∈ [0, 1]ψ1 (t) = ln (at

α + b) / ln (b) a > 0, b > 1, α ∈ (0, 1]ψ1 (t) = (at

α + b) / (b (atα + 1)) a > 0; b, α ∈ (0, 1]Table 1: Building Functions

Fitted Model

Why This Model


Correlation Model for Irish Wind Data

For this data set, the estimated correlation model is

C (h; u|β) =

ψ−11 (u) =

(.901 |u|1.544 + 1

)−1if h = 0

.968ψ1 (u) exp

(− .00134 ‖h‖

ψβ/2(u)

)otherwise

(5)

where ψ1 and ϕ1 in Table 1 are chosen and ‖h‖ ≤ 450km, |u| ≤ 3

ψ−11 (u) =

(.901 |u|1.544 + 1

)−1, temporal correlation, estimated

from data

ϕ (h) = .968 exp (−.00134 ‖h‖) , spatial correlation, borrowed fromHaslett and Raftery (1989) ([4])

β = 0.61, spatial-temporal interaction, estimated from data; γ = 1/2in ϕ1 was forced

Table of Functions

Why This Model




C (h; u|β) =

ψ−11 (u) =

(.901 |u|1.544 + 1

)−1if h = 0

.968ψ1 (u) exp

(− .00134 ‖h‖

ψβ/2(u)

)otherwise

(5)


ψ−11 (u) =

(.901 |u|1.544 + 1


from data



Table of Functions

Why This Model




C (h; u|β) =

ψ−11 (u) =

(.901 |u|1.544 + 1

)−1if h = 0

.968ψ1 (u) exp

(− .00134 ‖h‖

ψβ/2(u)

)otherwise

(5)


ψ−11 (u) =

(.901 |u|1.544 + 1


from data



Table of Functions

Why This Model


Why the Above Covariance Function?

The fitted model is composed of

Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 ifh = 0;C (h; 0|β) = ϕ (h) , otherwise. Note ϕ ∈ C∞

Purely Temporal Correlation: C (u; 0|β) = ψ−11 (u) . Note ψ1 /∈ C∞

Some reasons:

Intrinsic: instrument variations and highly irregular wind speedsGeneric: evidence from data (see Graphs)

Technical: choice of ϕ fits into the family

C (h; u) =σ2(

a ‖u‖2α + 1)βk/2 exp

− c ‖h‖2γ(a ‖u‖2α + 1

)βγ

Table of Functions


http://www.stat.purdue.edu/~chen418/study_research/Tilmann_evidence.pdf





Some reasons:



C (h; u) =σ2(


− c ‖h‖2γ(a ‖u‖2α + 1

)βγ

Table of Functions







Some reasons:

Intrinsic: instrument variations and highly irregular wind speeds

Generic: evidence from data (see Graphs)


C (h; u) =σ2(


− c ‖h‖2γ(a ‖u‖2α + 1

)βγ

Table of Functions







Some reasons:



C (h; u) =σ2(


− c ‖h‖2γ(a ‖u‖2α + 1

)βγ

Table of Functions







Some reasons:



C (h; u) =σ2(


− c ‖h‖2γ(a ‖u‖2α + 1

)βγ

Table of Functions




All covariance functions in Cressie and Huang (1999) ([1]) andGneiting (2002) ([2]) are FULLY SYMMETRIC

Lagrangian reference frame by Cox and Isham (1988) ([6])

C (h; u) = EVG (‖h− Vu‖) ; (h, u) ∈ R2 ×R

used to model dynamic environmental and atmospheric processes,where

G (r) = λ {D (P1, 1) ∩D (P2, 1) : ‖P1 − P2‖ = r} ;P1,P2 ∈ R3

and λ is the Lebesgue measure.

Perturbation to fully symmetric models

False claims of convexity lead to false PSD functions, as inCressie and Huang 1999 ([1])



All covariance functions in Cressie and Huang (1999) ([1]) andGneiting (2002) ([2]) are FULLY SYMMETRICLagrangian reference frame by Cox and Isham (1988) ([6])

C (h; u) = EVG (‖h− Vu‖) ; (h, u) ∈ R2 ×R


G (r) = λ {D (P1, 1) ∩D (P2, 1) : ‖P1 − P2‖ = r} ;P1,P2 ∈ R3







C (h; u) = EVG (‖h− Vu‖) ; (h, u) ∈ R2 ×R


G (r) = λ {D (P1, 1) ∩D (P2, 1) : ‖P1 − P2‖ = r} ;P1,P2 ∈ R3







C (h; u) = EVG (‖h− Vu‖) ; (h, u) ∈ R2 ×R


G (r) = λ {D (P1, 1) ∩D (P2, 1) : ‖P1 − P2‖ = r} ;P1,P2 ∈ R3





Generalized Theorem 1 and Proof: I

Theorem 1 (Generalized): A continuous, bounded, symmetric andintegrable function C (h; u) ,defined on Rk ×Rl ,is a covariance function ifand only if

Cω (u) =∫e−ih

′ωC (h; u) dh, u ∈ Rl (6)

is a covariance function for all most all ω ∈ Rk

Digested Proof: The extension is on u ∈ Rl .The arguments are just acombination of Fubini’s, Bochner’s theorems and the Dirichlet integral.(a) C ∈ R1

(Rk ×Rl

)⇒ C ∈ L1

(Rk ×Rl

)(whenever k + l ≥ 2) and

Cω ∈ L1(Rl),a.s. ω ∈ Rk .

(b) C ∈(Cb ∩ L1

) (Rk+l

)⇒ Cω ∈ Cac

(Rl), ∀ω ∈ Rk ;

(c) C ∈(Cb ∩ L1

) (Rk+l

)and the inversion formula imply that

Inversion Formular

Theorem 2 and Proof


Generalized Theorem 1 and Proof: II

Digested Proof (continued): C has F-inverse as

f (ω, τ) ∝∫∫

e−ih′ω−iτ′uC (h; u) dhdu ∝

∫e−iτ

′uCω (u) du (7)

where f =dFdλ

and F is a signed, bounded, measure on Rk+l such that

F << λ and λ is the Lebesgue measure on Rk+l . The last equality isjustified by Fubini’s theorem.(d) Now, Bochner’s theorem and Fubini’s together implies the validity ofthe theorem iff f ≥ 0 on Rk ×Rl and f ∈ L1

(Rk+l

). This completes the

proof.Inversion Formular


Theorem 2 and Proof: Multiply Dominating Factor

Theorem 2 (Generalized): Let k and l be nonnegative integers, and letσ2 > 0. Suppose that ϕ (t) , t ≥ 0, is a completely monotone function,and let ψ (t) , t ≥ 0, be a positive function with a completely monotonederivative. Then

C (h; u) =σ2

ψ(‖u‖2

)k/2 ϕ

‖h‖2

ψ(‖u‖2

) , (h; u) ∈ Rk ×Rl (8)

is a space-time covariance function.Digested Proof: (a) Multiply the target function by dominatingintegral factor. (Case Restricted) Assume ϕ ∈ R1

(Rk)and define

Ca (h; u) = exp(−a ‖u‖2

)C (h; u) (9)

By Theorem 1 (Generalized), (9) is PSD iff (6) is PSD.Theorem 1 and Proof


Theorem 2 and Proof: Plug in L-Representation of ϕ

(proof continued): (b) Plug the Laplace transform representation ofϕ into the integral. Notice that

ϕ (t) =∫[0,∞)

exp (−tr) dF (r) = L (F ) (10)

and ϕ ∈ R1(Rk), then ϕ is bounded and limt→0 ϕ (t) = 0.Thus F is a

bounded and F is right-continuous at 0 (Otherwise, take tn = n, rn = 1/nin (10)).

For notational simplicity, let ψu = ψ(‖u‖2

). Then by Fubini theorem,

Ca,ω (u) =∫e−ih

′ωCa (h; u) dh (11)

= exp(−a ‖u‖2

) σ2

ψk/2u

∫(0,∞)

dF (r)∫(0,∞)

fu,r ,ω (h) dh

where fu,r ,ω (h) = e−ih′ω exp

(−(r ‖h‖2

)/ψu

)Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 14 / 22

Theorem 2 and Proof: Obtain L-Representation of Ca,ω(proof continued): and (c1) that

Ca,ω (u) = ϕa,ω

(‖u‖2

), u ∈ Rk (12)

can be assumed, where

ϕa,ω (t) = σ2πk/2 exp (−at)∫(0,∞)

exp (−sψ (t)) dGω,a (s) ; t ≥ 0 (13)

and Gω,a is a nondecreasing, bounded function. This is just the Laplacetransform of the measure induced by composing the measures F byψ.Since Ca,ω is spherically symmetric and ϕa,ω is cm, Shoenberger’s theorem(Shoenberger 1938 [5]) asserts that Ca,ω (·) is PSD and so is each Ca byTheorem 1. Since lima→0+ Ca = C pointwise and C is continuous at 0, Cis PSD.

Theorem 2 and Proof 3


Theorem 2 and Proof: Still by Dominating Factor

(proof continued): (Case General) Given a cm function ϕ (t) , t ≥ 0,then hb (t) = exp (−bt) ϕ (t) , b > 0, t ≥ 0 is still cm (by Tonelli’stheorem and since exp (− (b+ r) t) is integrable and F in theL-representation of ϕ is a bounded measure). Further hb is dominated byexp (−bt) , hence hb ∈ L1 (R) and hb ∈ Rk when t = ‖h‖ .Thus

Cb (h; u) =σ2

ψ(‖u‖2

)k/2 exp

− b ‖h‖2

ψ(‖u‖2

) ϕ

‖h‖2

ψ(‖u‖2

) (14)

is PSD by the previous arguments. Again, limb→0+ Cb = C pointwise andC is continuous at 0, C is PSD.

Acknowledgements


Details On Equivalences and Spectral Representation

f positive definite ⇒ Gw = (f (wi − wj )) a positive-definite matrix⇒ a characteristic function (ch.f) hw (t) = exp

(−12t ′Gw t

)⇔

hw (s) =∫e−i 〈t ,s〉hw (t) dt a finite measure. Take a process Z (w)

(subtracted its mean function) with hw as its distribution.

when f ∈ L1(Rd+1

), F is absolute continuous and f ,F are inverse

Fourier transformations (modulo a constant). Based on the inversionformula

F (a, b] = limn→∞

12π

∫ T

−Tdt∫ b

ae−itsds

via Dirichlet integral lima→−∞,b→+∞∫ basin xxdx = π

Every psd function is equivalent to a double integral withnon-negative integrand.

PSD Functions

Proof of Theorem1 Proof of Theorem1 (II)


Proof of Bochner’s Theorem

Digested Proof (Feller [3], p58):∑ki ,j=1 f (wi − wj ) aia∗j ≥ 0⇒∫∫

f (α− β) ρ (α) ρ (β)dµαdµβ ≥ 0, ∀ρ ∈ L1

Fix x and ε > 0 and put ρ (α) = e−2ε|α|2e2π〈α−β,x 〉. Do transformationα− β = γ, α+ β = δ to obtain∫ (∫

e−ε|δ|2dµδ

)e−ε|γ|2 f (γ) e2πi 〈γ,x 〉dµγ ≥ 0 =⇒ fε (γ) =

e−ε|γ|2 f (γ) ∈ L1 and has non-negative F -inverse. Hence fε has spectralrepresentation and limε→0 fε = f completes the proof by weak andcomplete convergence.Covariance Functions


Details on Completely Monotone Functions

Theorem(Feller [3], p. 439): ϕ : (0,∞)→ (0,∞) is said to becompletely monotone iff (−1)n ϕn (t) ≥ 0, t > 0, n ≥ 0 iff

ϕ (t) =∫[0,∞)

exp (−tr) dF (r) = L (F )

with F being a measure on [0,∞].Digested Proof (Feller [3], p440).

ϕ∗ (s) = ϕ (a− as) = ∑(−a)n ϕn (a)

n!sn =⇒ ϕa (λ) = ϕ

(a− ae−λ/a) =

∑(−a)n ϕn (a)

n!e−nλ/a.And lima→∞ ϕa = ϕ ⇐⇒ lima→∞ Fa = F . ϕ is a

Laplace transform.Moreover under the above settings, it holds

F (x) = lima→∞ ∑n≤ax(−a)n ϕn (a)

n!the family of cm functions is closed under multiplication andcomposition, justified by induction via definition.

Construction of PDS Functions


Theorem 2 and Proof: Contour integral of fu,r ,ω(h)

(proof continued): (c) Derive the Laplace transform from Cω. LetD = 2diag {r/ψu , · · · , r/ψu}. Notice that D > 0 andfu,r ,ω (h) ∝ e−ih

′ω−h′Dh/2, then rectangle contour integral by Cauchyintegral formular shows

fu,r (ω) =∫(0,∞)

fu,r ,ω (h) dh =(

πrψu

)−k/2

exp

(−‖ω‖

2

4rψu

)and

Cω (u) = σ2 exp(−a ‖u‖2

)exp

(−‖ω‖

2

4rψu

) ∫(0,∞)

1r k/2 dF (r) (15)

The continuity of C0 (·) and (15) implies∫(0,∞)

1r k/2 dF (r) =

C0 (0)σ2πk/2 < ∞

Theorem 2 and Proof 4


References

Cressie, N., and Huang, H.-C. (1999), Classes of Nonseparable,Spatiotemporal Stationary Covariance Functions, JASA, 94, 1330-1340

Tilmann Gneiting (2002), Nonseparable, Stationary Covariance Functions forSpace-Time Data, JASA, Vol. 97, No. 458 (Jun., 2002), pp. 590-600

Feller, W. (1966), An Introduction to Probability Theory and Its Applications(Vol. II), New York: Wiley

John Haslett and Adrian E. Raftery (1989), Space-Time Modelling withLong-Memory Dependence: Assessing Ireland’s Wind Power, JRSS. Ser C(Applied Statistics), Vol. 38, No. 1(1989), pp. 1-50

Schoenberg, I. J. (1938), Metric Spaces and Completely MonotoneFunctions, Annals of Mathematics, 39, 811-841.

Cox, D. R., and Isham, V. (1988), A Simple Spatial-Temporal Model ofRainfall, in Proceedings of the Royal Society of London, Ser. A, 415, 317-328


Acknowledgements

Thank you for being so PATIENT!

Irish Wind

Thoerem2 and Proof 5


Documents

Stat 598 Presentation on Nonseperable, Stationary ...chen418/study_research/stat598... · Stat 598 Presentation on Nonseperable, Stationary Covariance Functions for Space-Time Data