61
university-logo Introduction Spatial Prediction Bayesian Spatial Prediction: BTG Benjamin Kedem Department of Mathematics & Inst. for Systems Research University of Maryland, College Park Victor De Oliveira, David Bindel, Boris and Sandra Kozintsev GWU, February 20, 2009 Benjamin Kedem Bayesian Spatial Prediction: BTG

Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Bayesian Spatial Prediction: BTG

Benjamin Kedem

Department of Mathematics & Inst. for Systems ResearchUniversity of Maryland, College Park

Victor De Oliveira, David Bindel, Boris and Sandra Kozintsev

GWU, February 20, 2009

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 2: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Outline

1 IntroductionStationary isotropic Gaussian random fields

2 Spatial PredictionOrdinary KrigingBTG

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 3: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

(•) Berger, De Oliveira, Sanso (2001). Objective BayesianAnalysis of Spatially Correlated Data. JASA, 96, 1361-1374.

(•) De Oliveira, Kedem, Short (1997). Bayesian Prediction ofTransformed Gaussian Random Fields. JASA, 92, 1422-1433.

(•) De Oliveira, Ecker (2002). Bayesian Hot Spot Detection inthe Presence of a Spatial Trend: Application to Total NitrogenConcentration in the Chesapeake Bay. Environmetrics, 13,85-101.

(•) Handcock, Stein (1993). A Bayesian Analysis of Kriging.Technometrics, 35, 403-410.

(•) Kedem, Fokianos (2002). Regression Models for TimeSeries Analysis, New York: Wiley.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 4: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

(•) Kozintsev, Kedem (2000). Generation of "Similar" ImagesFrom a Given Discrete Image. J. Comp. Graphical Stat., 2000,Vol. 9, No. 2, 286-302.

(•) Kozintseva (1999). Comparison of Three Methods of SpatialPrediction. M.A. Thesis, Department of Mathematics, Universityof Maryland, College Park.

(•) Bindel, De Oliveira, Kedem (1997). An implementation ofthe BTG spatial prediction model.http://www.math.umd.edu/bnk/btg_page.html

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 5: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spatial/temporal geostatistical data often display:

Non-Gaussian skewed sampling distributions.Positive continuous data.Heavy right tails.Bounded support.Small data sets observed irregularly (gaps).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 6: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spatial/temporal geostatistical data often display:

Non-Gaussian skewed sampling distributions.Positive continuous data.Heavy right tails.Bounded support.Small data sets observed irregularly (gaps).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 7: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spatial/temporal geostatistical data often display:

Non-Gaussian skewed sampling distributions.Positive continuous data.Heavy right tails.Bounded support.Small data sets observed irregularly (gaps).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 8: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spatial/temporal geostatistical data often display:

Non-Gaussian skewed sampling distributions.Positive continuous data.Heavy right tails.Bounded support.Small data sets observed irregularly (gaps).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 9: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spatial/temporal geostatistical data often display:

Non-Gaussian skewed sampling distributions.Positive continuous data.Heavy right tails.Bounded support.Small data sets observed irregularly (gaps).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 10: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Possible remedies:

Bayesian Transformed Gaussian (BTG): A Bayesianapproach combined with parametric families of nonlineartransformations to Gaussian data.BTG provides a unified framework for inference andprediction/interpolation in a wide variety of models,Gaussian and non-Gaussian.Will describe BTG and illustrate it using spatial andtemporal data.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 11: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Possible remedies:

Bayesian Transformed Gaussian (BTG): A Bayesianapproach combined with parametric families of nonlineartransformations to Gaussian data.BTG provides a unified framework for inference andprediction/interpolation in a wide variety of models,Gaussian and non-Gaussian.Will describe BTG and illustrate it using spatial andtemporal data.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 12: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Possible remedies:

Bayesian Transformed Gaussian (BTG): A Bayesianapproach combined with parametric families of nonlineartransformations to Gaussian data.BTG provides a unified framework for inference andprediction/interpolation in a wide variety of models,Gaussian and non-Gaussian.Will describe BTG and illustrate it using spatial andtemporal data.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 13: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Let Z (s), s ∈ D ⊂ Rd , be a spatial process or a random field.

A random field Z (s) is Gaussian if for all s1, ..., sn ∈ D, thevector (Z (s1), ..., Z (sn)) has a multivariate normal distribution.

Z (s) is (second order) stationary when for s, s + h ∈ D wehave

(•) E(Z (s)) = µ,

(•) Cov(Z (s + h), Z (s)) ≡ C(h).

The function C(·) is called the covariogram or covariance

function.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 14: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

We shall assume that C(h) depends only on the distance ‖h‖between the locations s + h and s but not on the direction of h.

In this case the covariance function as well as the process arecalled isotropic.

The corresponding isotropic correlation function is given byK (l) = C(l)/C(0), where l is the distance between points.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 15: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Useful special case: Matérn correlation

Kθ(l) =

1

2θ2−1Γ(θ2)

(l

θ1

)θ2κθ2

(l

θ1

)if l 6= 0

1 if l = 0

where θ1 > 0, θ2 > 0, and κθ2 is a modified Bessel function ofthe third kind of order θ2.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 16: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Matérn (θ1 = 8, θ2 = 3).

2040

6080

100120

X20

40

60

80

100

120

Y

-4-3

-2-1

01

23

Z

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 17: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spherical correlation:

Kθ(l) =

1− 3

2

( lθ

)+ 1

2

( lθ

)3if l ≤ θ

0 if l > θ

where θ > 0 controls the correlation range.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 18: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Spherical (θ = 120).

2040

6080

100120

X20

40

60

80

100

120

Y

-2-1

01

2Z

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 19: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Exponential correlation:

Kθ(l) = exp(lθ2 log θ1), θ1 ∈ (0, 1), θ2 ∈ (0, 2]

Exponential (θ1 = 0.5, θ2 = 1).

2040

6080

100120

X20

40

60

80

100

120

Y

-4-2

02

46

Z

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 20: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Rational quadratic:

Kθ(l) =(

1 +2

θ21

)−θ2

θ1 > 0, θ2 > 0.

Rational quadratic (θ1 = 12, θ2 = 8).

2040

6080

100120

X20

40

60

80

100

120

Y

-4-2

02

4Z

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 21: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Clipped, at 3 levels, realizations from Gaussian random fields.Left: Matérn (8,3). Right: spherical (120).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 22: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Stationary isotropic Gaussian random fields

Clipped, at 3 levels, realizations from Gaussian random fields.Left: exponential (0.5,1). Right: rational quadratic (12,8).

www.math.umd.edu/-bnk/bak/generate.cgi?4Kozintsev (1999), Kozintsev and Kedem (2000).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 23: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Outline

1 IntroductionStationary isotropic Gaussian random fields

2 Spatial PredictionOrdinary KrigingBTG

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 24: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Ordinary Kriging.

Given the data

Z ≡ (Z (s1), . . . , Z (sn))′

observed at locations s1, . . . , sn in D, the problem is topredict (or estimate) Z (s0) at location s0 using the best linearunbiased predictor (BLUP) obtained by minimizing

E(Z (s0)−

n∑i=1

λiZ (si))2 subject to

n∑i=1

λi = 1

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 25: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Define

1 = (1, 1, . . . , 1)′, 1× n vector

c =(C(s0 − s1), . . . , C(s0 − sn)

)′C =

(C(si − sj)

), i , j = 1, ..., n

λ = (λ1, λ2, . . . , λn)′

Then

λ = C−1(

c +1− 1′C−1c

1′C−111)

.

The ordinary kriging predictor is then

Z (s0) = λ′Z.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 26: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Define,

m =1− 1′C−1c

1′C−11and the kriging variance

σ2k (s0) = E

(Z (s0)− Z (s0)

)2= C(0)− λ

′c + m.

Under the Gaussian assumption,

Z (s0)± 1.96σk (s0)

is a 95% prediction interval for Z (s0). For non-Gaussian fieldsthis may not hold.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 27: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Bayesian Spatial Prediction: The BTG Model

RF Z (s), s ∈ D observed at s1, . . . , sn ∈ D. Parametricfamily of monotone transformations

G = gλ(.) : λ ∈ Λ

(?) Assumption: Z (.) can be transformed into a Gaussianrandom field by a member of G.A useful parametric family of transformations often used inapplications to ‘normalize’ positive data is the Box-Cox (1964)family of power transformations,

gλ(x) =

xλ−1

λ if λ 6= 0log(x) if λ = 0

.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 28: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

For some unknown ‘transformation parameter’ λ ∈ Λ,gλ(Z (s)), s ∈ D is a Gaussian random field with

Egλ(Z (s)) =

p∑j=1

βj fj(s),

covgλ(Z (s)), gλ(Z (u)) = τ−1Kθ(s, u),

Regression parameters: β = (β1, . . . , βp)′

Covariates: f(s) = (f1(s), . . . , fp(s))

Variance: τ−1 = vargλ(Z (s))Simplifying assumption: Isotropy,

Kθ(s, u) = Kθ(||s− u||), θ = (θ1, . . . , θq) ∈ Θ ⊂ Rq

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 29: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Data: Zobs = (Z1,obs, . . . , Zn,obs)

gλ(Zi,obs) = gλ(Z (si)) + εi ; i = 1, . . . , n,

ε1, . . . , εn are i.i.d. N(0, ξτ ).

Parameters: η = (β, τ, ξ, θ, λ).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 30: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Prediction problem:

Predict Z0 = (Z (s01), . . . , Z (s0k )) from the predictive densityfunction, defined by

p(zo|zobs) =

∫Ω

p(zo,η|zobs)dη

=

∫Ω

p(zo|η, zobs)p(η|zobs)dη,

where Ω = Rp × (0,∞)2 ×Θ× Λ.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 31: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Notation: For a = (a1, . . . , an), we write

gλ(a) ≡ (gλ(a1), . . . , gλ(an)).

The Likelihood:

L(η; zobs) =( τ

) n2∣∣∣Ψξ,θ

∣∣∣− 12 exp

−τ

2Q

Jλ,

Q =(

gλ(zobs)− Xβ

)′Ψ−1

ξ,θ

(g

λ(zobs)− Xβ

).

X n × p design matrix, Xij = fj(si).Ψξ,θ = Σθ + ξI, n × n matrix.Σθ;ij = Kθ(si , sj).I identity matrix.Jλ =

∏ni=1 |g′(zi,obs)|, the Jacobian.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 32: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

The Prior

Insightful arguments in Box and Cox(1964), De Oliveira,Kedem, Short (1997), as well as practical experience lead us tothe prior

p(η) ∝ p(ξ)p(θ)p(λ)

τJpnλ

,

where p(ξ), p(θ) and p(λ) are the prior marginals of ξ, θ and λ,respectively, which are assumed to be proper.

Unusual prior: it depends on the data through the Jacobian.

For more on prior selection see Berger, De Oliveira, Sansó(2001).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 33: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Simplifying assumption: No measurement noise (ξ = 0).

gλ(Zi,obs) = gλ(Z (si)), i = 1, . . . , n,

p(β, τ, θ, λ) ∝ p(θ)p(λ)

τJp/nλ

(1)

η = (β, τ, θ, λ)′.

Also, writez = zobs

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 34: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

The Posterior

p(η|z) = p(β, τ, θ, λ|z) = p(β, τ |θ, λ, z)p(θ, λ|z).

To get the first factor:

(β|τ,θ, λ, z) ∼ Np(βθ,λ,1τ(X′Σ−1

θX)−1)

(τ |θ, λ, z) ∼ Ga(n − p

2,

2qθ,λ

)

whereβθ,λ = (X′Σ−1

θX)−1X′Σ−1

θg

λ(z)

qθ,λ = (gλ(z)− Xβθ,λ)′Σ−1

θ(g

λ(z)− Xβθ,λ).

and we get Normal-Gamma:

p(β, τ |θ, λ, z) = p(β|τ,θ, λ, z)p(τ |θ, λ, z)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 35: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

To get the second factor:

p(θ, λ|z) ∝

|Σθ|−1/2|X′Σ−1

θX|−1/2q

− n−p2

θ,λJ

1− pn

λ p(θ)p(λ)

In addition to the joint posterior distribution p(η|z) derivedabove, the predictive density p(zo|z) also requires p(zo|η, z).We have

p(zo|η, z) = (τ

2π)k/2|Dθ|

−1/2k∏

j=1

|g′λ(zoj)|

×exp−τ

2(g

λ(zo)−Mβ,θ,λ)′D−1

θ(g

λ(z)−Mβ,θ,λ)

where Mβ,θ,λ, Dθ are known.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 36: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

We now have the integrand p(zo|η, z)p(η|z) needed for p(zo|z).By integrating out β and τ we obtain the simplified form of thepredictive density:

p(zo|z) =

∫Λ

∫Θ

p(zo|θ, λ, z)p(θ, λ|z)dθdλ

=

∫Λ

∫Θ p(zo|θ, λ, z)p(z|θ, λ)p(θ)p(λ)dθdλ∫

Λ

∫Θ p(z|θ, λ)p(θ)p(λ)dθdλ

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 37: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

where

p(zo|θ, λ, z) =Γ(n−p+k

2 )∏k

j=1 |g′λ(zoj)|

Γ(n−p2 )πk/2|qθ,λCθ|1/2

×[1 + (gλ(zo)−mθ,λ)′(qθ,λCθ)−1

×(gλ(zo)−mθ,λ)]−

n−p+k2

and from Bayes theorem,

p(z|θ, λ) ∝ |Σθ|−1/2|X′Σ−1

θX|−1/2q

− n−p2

θ,λJ

1− pn

λ

where mθ,λ, Cθ are known.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 38: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

BTG Algorithm: Predictive Density Approximation

1. Let S = z(j)o : j = 1, . . . , r be the set of values obtained

by discretizing the effective range of Z0.

2. Generate independently θ1, . . . ,θm i.i.d. ∼ p(θ) andλ1, . . . , λm i.i.d. ∼ p(λ).

3. For zo ∈ S, the approximation to p(zo|z) is given by

pm(zo|z) =

∑mi=1 p(zo|θi , λi , z)p(z|θi , λi)∑m

i=1 p(z|θi , λi)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 39: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

p(zo|θ, λ, z) and p(z|θ, λ) given above.

The point predictor is the median of the estimated predictivedistribution:

(?) Z0 = Median of (Z0|Z)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 40: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

tkbtg Interface Layout

Software: tkbtg application. Hybrid of C++, Tcl/Tk, andFORTRAN 77 (Bindel et al (1997)).www.math.umd.edu/-bnk/btg_page.html

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 41: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 42: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Example: Spatial Rainfall Prediction

Rain gauge positions and weekly rainfall totals in mm, Darwin,Australia, 1991.

x

y

0 2 4 6 8 10 12

02

46

810

12

29.55 41.85

26.7633.98

31.27

54.35 30.57

22.8266.76 33.68 19.31

23.6963.1 46.06

24.47

28.9868.2526.83 18.24

31.14

21.717.72

29.92 23.6

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 43: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

1. Use the Box-Cox transformation family.2. λ ∼ Unif(−3, 3).3. m = 500.4. Correlation: Matérn and exponential.5. No covariate information. Assume constant regression:Egλ(Z (s)) = β1.6. Data apparently not normal.

10 20 30 40 50 60 70

0.0

0.01

0.02

0.03

0.04

Total Rainfall in mm

Mean=33.94

Median=29.73

SD=15.06

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 44: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Results of Cross Validation From 23 Observations

No. z z 95% PI1 29.55 33.74 (10.70, 56.78)2 41.85 34.32 (15.29, 53.35)3 26.76 36.71 (20.93, 52.49)4 33.98 33.39 (18.49, 48.29)5 31.27 32.99 (9.53, 56.45)6 54.35 39.13 (16.46, 61.79)7 30.57 24.45 (15.40, 33.59)8 22.82 23.09 (11.52, 34.66)9 66.76 64.12 (28.25, 100)

10 33.68 35.16 (18.01, 52.30)11 19.31 24.51 (15.62, 33.40)12 23.69 26.45 (15.35, 37.54)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 45: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

No. z z 95% PI13 63.10 72.07 (44.14, 100)14 46.06 40.60 (18.58, 62.63)15 24.47 22.32 (14.00, 30.63)16 28.98 21.62 (13.43, 29.81)17 68.25 46.54 (19.25, 73.84)18 26.83 29.52 (16.57, 42.46)19 18.24 19.00 (10.79, 27.21)20 31.14 37.36 (20.33, 54.39)21 21.70 22.97 (14.71, 31.22)22 17.72 22.69 (11.64, 33.74)23 29.92 26.56 (12.21, 40.91)24 23.60 21.83 (10.85, 32.81)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 46: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Spatial prediction and contour maps from the Darwin datausing Matérn correlation.

0

2

4

6

8

10

X

0

2

4

6

8

10

Y

1020

3040

5060

7080

Z

x

y

0 2 4 6 8 10 12

02

46

810

12

20

25

30

30

30

30

30

35 35

40

40

45

50

55

60

65

70

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 47: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Spatial prediction and contour maps from the Darwin datausing exponential correlation.

0

2

4

6

8

10

X

0

2

4

6

8

10

Y

1020

3040

5060

7080

Z

x

y

0 2 4 6 8 10 12

02

46

810

12

20

25

25

25

30

30

35 3540

40

40

45

45

50

50

55

55

6060

65

70

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 48: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Predictive densities, 95% PI’s, and cross-validation: Predictinga true value from the remaining 23 observations using Matérncorrelation. The vertical line marks true values.

x

p(x)

0 20 40 60 80 100

0.0

0.04

0.08

0.12

True: 41.85Median: 34.31(15.60, 53.02)

x

p(x)

0 20 40 60 80 100

0.0

0.04

0.08

0.12

True: 63.10Median: 70.76(41.52, 100)

x

p(x)

0 20 40 60 80 100

0.0

0.04

0.08

0.12

True: 28.98Median: 22.1(13.26, 30.94)

x

p(x)

0 20 40 60 80 100

0.0

0.04

0.08

0.12

True: 21.70Median: 23.05(14.84, 31.26)

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 49: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

BTG vs Kriging and Trans-Gaussian kriging(Kozintseva (1999)).

Cross validation results using artificial data on 50× 50 grid.Data obtained by transforming a Gaussian (0,1) RF usinginverse Box-Cox transformation.In Kriging and TG kriging λ, θ, were known. Not in BTG (!)λ = 0: Log-Normal.λ = 1: Normal.λ = 0.5: Between Normal and Log-Normal.In most cases BTG has more reliable but larger predictionintervals.BTG predicts at the original scale. TG kriging does not.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 50: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Matérn(1,10)λ 0 0.5 1

KRG MSE 68397.48 7.15 0.58TGK MSE 55260.90 7.08 0.58BTG MSE 64134.30 7.31 0.56KRG AvePI 2.42 2.51 2.42TGK AvePI 291.80 8.21 2.42BTG AvePI 330.68 10.23 2.87KRG % out 100% 48% 6%TGK % out 18% 8% 6%BTG % out 12% 6% 6%

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 51: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Exponential(e−0.03, 1)λ 0 0.5 1

KRG MSE 12212.32 1.83 0.13TGK MSE 11974.73 1.84 0.13BTG MSE 12520.70 1.89 0.14KRG AvePI 1.45 1.43 1.45TGK AvePI 267.92 5.24 1.45BTG AvePI 466.69 6.10 1.63KRG % out 98% 64% 2%TGK % out 20% 4% 2%BTG % out 6% 2% 2%

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 52: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Application of BTG to Time Series Prediction.

Short time series observed irregularly.Set: s = (x , y) = (t , 0).Can predict/interpolate as in state space prediction:k–step prediction forward, backward, and “in the middle”.Example 1: Monthly data of unemployed women 20 yearsof age and older, 1997–2000. Data source: Bureau ofLabor Statistics. N = 48.Example 2: Monthly airline passenger data, 1949–1960.Data source: Box-Jenkins (1976). Use only N = 36 out of144 observations, t = 51, ..., 86.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 53: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Example: Prediction of Monthly number ofunemployed women (Age ≥ 20), 1997–2000. Data inhundreds of thousands.

Cross validation and 95% PI’s. Observations at timest = 12, 13, 36 are outside their 95% PI’s. N = 48− 1 = 47.

t

No.

Une

mpl

oyed

Wom

en

0 10 20 30 40

1015

2025

3035

TruePredictedLowerUpper

Unemployed Women 1997--2000

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 54: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Forward and backward one and two step prediction in theunemployed women series. In 2-step higher dispersion.

x

p(x)

10 20 30 40 50

0.0

0.05

0.10

0.15

0.20

True: 18.34Median: 21.78(17.24, 26.33)

One step forward

x

p(x)

10 20 30 40 50

0.0

0.05

0.10

0.15

0.20

True: 18.34Median: 22.20(16.36, 28.05)

Two step forward

x

p(x)

10 20 30 40 50

0.0

0.05

0.10

0.15

0.20

True: 28.98Median: 26.17(21.94, 30.40)

One step backward

x

p(x)

10 20 30 40 50

0.0

0.05

0.10

0.15

0.20

True: 28.98Median: 24.69(19.77, 29.61)

Two step backward

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 55: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Example: Prediction of No. of Airline Passengers.

Time series of monthly international airline passengers inthousands, January 1949-December 1960. N=144. Seasonaltime series.

t

No.

Pas

seng

ers

0 20 40 60 80 100 120 140

100

200

300

400

500

600

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 56: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

BTG cross validation and prediction intervals for the monthlyairline passengers series, t = 51, ..., 86, using Matérncorrelation. Observations at t = 62, 63 are outside the PI.N = 36− 1 = 35.

t

No.

Pas

seng

ers

50 60 70 80

150

200

250

300

350

400

TruePredictedLowerUpper

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 57: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Application to Rainfall: Heuristic Argument (Kedemand Chiu, PNAS, 1987.)

Let Xn represent the area average rain rate over a region suchthat

Xn = mXn−1 + λ + εn, n = 1, 2, 3, · · · ,

where the noise εn is a martingale difference. It can beargued that for

Xn → LogNormal, n →∞,

we need m → 1− and λ → 0+.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 58: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

The monotone increase in m (Curve a) and the monotonedecrease in λ (Curve b) as a function of the square root of thearea. Source: Kedem and Chiu(1987).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 59: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

This suggests that the lognormal distribution as a model foraverages or rainfall amounts over large areas or long periods.

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 60: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

It is interesting to obtain the posterior p(λ | z) of λ, thetransformation parameter in the Box-Cox family, given the data,where the data are weekly rainfall totals from Darwin, Australia.

With a uniform prior for λ, the medians of p(λ | z) in 5 differentweeks are:

Week 1 median = − 0.45 (to the left of 0).Week 2 median = 0.45 (to the right of 0).Week 3 median = 0.15 (Not far from 0).Week 4 median = 0.20 (Not far from 0).Week 5 median = 0.95 (Close to 1).

Benjamin Kedem Bayesian Spatial Prediction: BTG

Page 61: Bayesian Spatial Prediction: BTGbnk/BTG.beam.pdf · BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non-Gaussian

university-logo

IntroductionSpatial Prediction

Ordinary KrigingBTG

Weekly posterior p(λ | z) of λ given rainfall totals from Darwin,Australia, for 5 different weeks.

(a)

lambda

dens

ity

-3 -2 -1 0 1 2 3

0.0

0.5

1.0

1.5

2.0

MAE = -0.45

(b)

lambda

dens

ity

-3 -2 -1 0 1 2 3

0.0

0.5

1.0

1.5

2.0

MAE = 0.45

(c)

lambda

dens

ity

-3 -2 -1 0 1 2 3

0.0

0.5

1.0

1.5

2.0

MAE = 0.15

(d)

lambda

dens

ity

-3 -2 -1 0 1 2 3

0.0

0.5

1.0

1.5

2.0

MAE = 0.2

(e)

lambda

dens

ity

-3 -2 -1 0 1 2 3

0.0

0.5

1.0

1.5

2.0

MAE = 0.95

Benjamin Kedem Bayesian Spatial Prediction: BTG