View
14
Download
0
Category
Preview:
Citation preview
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More
Spatial Statistics with Image Analysis
Johan Lindstrom1
1Mathematical StatisticsCentre for Mathematical Sciences
Lund University
LundOctober 11, 2018
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 1/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More
Outline
Spatial Statistics with Image AnalysisBayesian statisticsHierarchical modellingEstimation Procedures
Spatial StatisticsStochastic FieldsGaussian Markov Random FieldsImage ReconstructionEnvironmental Data
Non-Gaussian DataExamples
Corrupted PixelsNDVI
Learn more
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 2/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Spatial Statistics
Many things vary in space and obsevations maydepend on what happens at nearby locations. Tomodell and analys such data we need spatialdependence.
Spatial Interpolation
Given observations at some locations (pixels),y(ui), i = 1 . . .nwe want to make statements about the value atunobserved location(s), x(u0).
Stationary Stochastic Processes (FMSF10)in 2+ dimensions!
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 3/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Bayesian modelling
We assume that there is some unknown truth, that wewould like to find out about. This “reality” can bemeasured, usually with measurement variation, and oftenonly partially.
Bayesian modelling
A Bayesian model consists ofI A prior, “a priori”, model for reality, x, given by the
probability density π(x).I A conditional model for data, y, given reality, with
density p(y|x).
The prior can be expanded into several layers creating aBayesian hierarchical model.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 4/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Bayes’ Formula
How should the prior and data model be combined to makestatements about the reality x, given observations of y?
Bayes’ Formula
p(x|y) = p(y|x)π(x)p(y)
=p(y|x)π(x)∫
x′∈Ω p(y|x′)π(x′) dx′
p(x|y) is called the posterior, or “a posteriori”, distribution.
Often, only the proportionality relation
p(x|y) ∝ p(x,y) = p(y|x)π(x)
is needed, when seen as a function of x.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 5/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Hierarchical Models
I We often have some prior knowledge of the reality.I Given knowledge of the true reality, what can we say
about images and other data?I Construct a model for observations given that we know
the truth.I Given data, what can we say about the unknown
reality?This is the inverse problem.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 6/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Bayesian hierarchical modelling (BHM)
A hierarchical model is constructed by systematicallyconsidering components/features of the data, and how/whythese features arise.
Bayesian hierarchical modelling
A Bayesian hierarchical model typically consists of (at least)
Data model, p(y|x): Describing how observations ariseassuming known latent variables x.
Latent model, p(x|θ): Describing how the latent variables(reality) behaves, assuming known parameters.
Parameters, π(θ): Describing our, sometimes vauge, priorknowledge of the parameters.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 7/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Bayesian statistics Hierarchical Estimation
Estimation Procedures
Maximum A Posteriori (MAP): Maximise the posteriordistribution p(x|y) with respect to x.
I Standard optimisation methodsI Specialised procedures, using the model
structure
Simulation: Simulate samples from the posteriordistribution p(x|y). Estimate statisticalproperties from these samples. The samples canbe seen as representative “possible realities”,given the available data.
I Markov chain Monte Carlo (MCMC)I Gibbs sampling
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 8/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Image Reconstruction
Spatial Interpolation
Given observations at some locations (pixels),y(ui), i = 1 . . .nwe want to make statements about the value atunobserved location(s), x(u0).
The typical model consists of a latent Gaussian field
x ∈ N (μ,Σ) ,
observed at locations ui, i = 1, . . . ,n, with additive Gaussiannoise (nugget or small scale variability)
yi = x(ui)+ εi εi ∈ N(
0,σ2ε
).
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 9/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Stochastic Fields
To perform the reconstruction (interpolation) we need amodel for the spatial dependence between locations(pixels).
1. Assume a latent Gaussian field
x ∈ N (μ,Σ) .
2. Assume a regresion model for μ = Bβ.
3. Assume a parametric (stationary) model for thedependence (covariance)
Σi,j = C(x(ui), x(uj)) = r(ui,uj;θ) = r(∥∥ui − uj
∥∥ ;θ).
r(ui,uj;θ) is called the covariance function.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 10/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Matern covariances functions
I One of the most common families of covariancefunctions is named after Bertil Matern, who worked forStatens Skogsforskningsinstitut (Forest Research Institute of Sweden).
I Variance σ2 > 0, scale parameter κ > 0 and shapeparameter ν > 0
rM(h) =σ2
Γ(ν)2ν−1 (κ‖h‖)νKν(κ‖h‖), h ∈ Rd,
I A measure of the range is given by ρ =√
8ν/κ.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 11/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
0 20 40 60
0
0.2
0.4
0.6
0.8
1
nu=0.5
0 20 40 60
0
0.2
0.4
0.6
0.8
1
nu=1.0
0 20 40 60
0
0.2
0.4
0.6
0.8
1
nu=2.0
0 20 40 60
0
0.2
0.4
0.6
0.8
1
nu=4.0
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 12/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
nu=0.5
5 10 15 20 25 30
5
10
15
20
25
30
nu=1.0
5 10 15 20 25 30
5
10
15
20
25
30
nu=2.0
5 10 15 20 25 30
5
10
15
20
25
30
nu=4.0
5 10 15 20 25 30
5
10
15
20
25
30
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 13/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
A local model
Instead of specifying the covariance function we couldconsider the local behaviour of pixels. A popular model isthe conditional autoregressive, CAR(1) model.
xij =1
4 + κ2
(xi−1,j + xi+1,j + xi,j−1 + xi,j+1
)+ ε,
ε ∈ N(
0,1τ2
).
This corresponds to a model for x where
x ∈ N(
0,Q−1),
where Q is called the precision matrix
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 14/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Matern covariances
The Matern covariance family
The covariance between two points at distance ‖h‖ is
rM(h) =σ2
Γ(ν)2ν−1 (κ‖h‖)νKν(κ‖h‖)
Fields with Matern covariances are solutions to a StochasticPartial Differential Equation (SPDE) (Whittle, 1954),
(κ2 −Δ
)α/2x(u) =W(u).
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 15/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Lattice on R2
Order α = 1 (ν = 0):
κ2
1
︸ ︷︷ ︸(C)
+
−1−1 4 −1
−1
︸ ︷︷ ︸≈−Δ (G)
Order α = 2 (ν = 1):
κ4
1
︸ ︷︷ ︸(C)
+2κ2
−1−1 4 −1
−1
︸ ︷︷ ︸≈−Δ (G)
+
12 −8 2
1 −8 20 −8 12 −8 2
1
︸ ︷︷ ︸≈Δ2 (G2=GC−1G)
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 16/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Spatial models for data
GMRF representations of SPDEs can be constructed foroscillating, anisotropic, non-stationary, non-separablespatio-temporal, and multivariate fields on manifolds.
(κ2 −Δ)(τ x(u)) =W(u), u ∈ Rd
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 17/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Spatial models for data
GMRF representations of SPDEs can be constructed foroscillating, anisotropic, non-stationary, non-separablespatio-temporal, and multivariate fields on manifolds.
(κ2 −Δ)(τ x(u)) =W(u), u ∈ Ω
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 17/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Spatial models for data
GMRF representations of SPDEs can be constructed foroscillating, anisotropic, non-stationary, non-separablespatio-temporal, and multivariate fields on manifolds.
(κ2 eiπθ −Δ)(τ x(u)) =W(u), u ∈ Ω
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 17/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Spatial models for data
GMRF representations of SPDEs can be constructed foroscillating, anisotropic, non-stationary, non-separablespatio-temporal, and multivariate fields on manifolds.
(κ2u +∇ ·mu −∇ ·Mu∇)(τux(u)) =W(u), u ∈ Ω
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 17/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Image Reconstruction II
Model with observations, y, and latent field, x,
y|x ∈ N(
Ax,σ2I)
x ∈ N(μ,Q−1
).
and Q = κ2C + G or Q = κ4C + 2κ2G + GC−1G.
Interpolation using a GMRF
E (x|y) = μ+1σ2 Q−1
x|y A> (y − Aμ)
V (x|y) = Q−1x|y =
(Q +
1σ2 A>A
)−1
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 18/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Image Reconstruction
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 19/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Global Temperature — Data
January 2003 July 2003
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 20/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Global Temperature — Reconstruction
Global mean: 15◦C.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 21/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Satellite Data — Vegetation
January 1999
July 1999
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 22/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Fields GMRF Reconstruction Environmental
Satellite Data — Trend in Vegetation David Bolin
K2 Estimate
−0.4
−0.2
0
0.2
0.4
Independent estimates
K2 Estimate
−0.3
−0.2
−0.1
0
0.1
Correlated estimates
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 23/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Examples
Non-Gaussian Data
Bayesian hierarchical modelling
A Bayesian hierarchical model typically consists of (at least)
Data model, p(y|x): Describing how observations ariseassuming known latent variables x.
Latent model, p(x|θ): x ∈ N(μ,Q−1
).
Parameters, π(θ)
So far we have assumed Gaussian observations
y|x ∈ N(
Ax,σ2I)
However we could (almost) as easily handle
yi|x ∈ F (g(Ax); θ)
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 24/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Examples
Larynx Cancer — Count data
Given counts of larynx cancer cases, yi, and population ineach region, Ei, we want to estimate the risk of cancer.
Counts of Larynx Cancer Population (truncated)
yi|xi ∈ Po (Ei exp(xi))
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 25/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Examples
Insurance Claims — Count data Oscar Tufvesson
Given the number of insurance claims, yi, we want toestimate the risk of an accident.
0,7500 1,5000
exp(X*beta)0,7500 1,5000
exp(u)
yi|ηi ∈ Po (Ei exp(ηi))
η = Bβ+ x
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 26/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Examples
Parana Rainfall — Positive data
January Precip.
5
10
15
20
January log(Precip.)
1
2
3
0 10 200
0.05
0.1
0.15
Gamma
Y5 10 15
5
10
15
20
1 2 30
0.5
1
Normal
log(Y)1 2 3
1
2
3
yi|xi ∈ Γ(
b,exi
b
)
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 27/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More Examples
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 28/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Image Reconstruction — Corrupted Pixels
I Typically we don’t know which pixels are bad.I A better model is then
I Assume an underlying image, x.I Assume an indicator image for bad pixels, z.I Given the indicator we either observe the correct pixel
value from x or noise.
I Use Bayes’ formula to compute the distribution for theunknown image (and indicator) given observations andparameters.
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 29/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Image Reconstruction — Corrupted pixels
image reconstruction
bad pixels − estimatebad pixels
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 30/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Normalized difference vegetation index (NDVI)
NDVI =RNIR − RRED
RNIR + RRED
I RRED is the amount of reflected red light (0.58−0.68 μm)I RNIR is the amount of reflected near-infrared light
(0.72− 1.00 μm)Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 31/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Smoothed version of the NDVI Data
Smooth the data to fill in missing values and remove noisedue to cloud cover, etc.
1986 1988 1990
140
160
180
200
Important ecological questions:I Plant phenology (start and end of season)I Plant productivity (integral)
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 32/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Smoothing of Satellite Based Vegetation Measurements
NDVI May 1993 Smooth
1991 1992 1993 1994 1995 1996
140
160
180
200
220
240
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 33/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More NDVI
Smoothing of Satellite Based Vegetation Measurements
1991 1992 1993 1994 1995 1996 1997
140
160
180
200
220
240
Obs
Gaussian
Gaussian 1D
Nig 1D
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 34/35
Spatial Statistics Spatial Non-Gaussian Corrupted Pixels More
Learn more!
What?Spatial statistics with image analysis, FMSN20
When?HT2-2018, October–December
Where?Information and Matlab files will be available atwww.maths.lth.se/matstat/kurser/fmsn20masm25/
(currently the 2017 webpage, updated soon)
Who?Lecturer: Johan Lindstrom
MH:319
Johan Lindstrom - johanl@maths.lth.se Spatial Statistics 35/35
Recommended