View
227
Download
0
Category
Preview:
Citation preview
Nonparametric inference in hidden Markov and related models
Roland Langrock, Bielefeld University
Roland Langrock | Bielefeld University 1 / 47
Introduction and motivation
Roland Langrock | Bielefeld University 2 / 47
Figure: Haggis (the dish).
Roland Langrock | Bielefeld University 3 / 47
Figure: Wild Haggis (Dux magnus gentis venteris saginati).
Roland Langrock | Bielefeld University 4 / 47
Introducing hidden Markov models using Wild Haggis movement
simulated movement track
●●
●●
●●
●●●
●
●
●
● ● ●
●
●●
●●
●● ●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●●
●●
●●●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●
●●
●
●●
●
●●
●●
●
●●●
●● ●
●●●
●
●
●●●● ●
●
Pr(St = j | St−1 = j) = 0.95 for j = 1, 2
where St : state at time t
Roland Langrock | Bielefeld University 5 / 47
Introducing hidden Markov models using Wild Haggis movement
simulated movement track
●●
●●
●●
●●●
●
●
●
● ● ●
●
●●
●●
●● ●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●●
●●
●●●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●
●●
●
●●
●
●●
●●
●
●●●
●● ●
●●●
●
●
●●●● ●
●
0 5 10 15 20 25 30
0.0
0.1
0.2
0.3
0.4
step length distributions
step length
dens
ity
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
turning angle distributions
turning angle
dens
ity
Pr(St = j | St−1 = j) = 0.95 for j = 1, 2,
where St : state at time t
Roland Langrock | Bielefeld University 6 / 47
Examples of HMM-type models/doubly stochastic processes
St−1 0St 0 St+1
Xt−1 0Xt 0 Xt+1
. . . . . . (hidden)
(observed)
hidden Markov models (HMMs)
(general) state-space models
Markov-switching regression
Cox point processes
In each case two components:
an observable state-dependent process1.) in animal movement: e.g. step lengths & turning angles2.) in financial time series: some economic indicator, e.g. GDP values3.) in disease progression: e.g. blood samples
a latent (nonobservable) state process/system processin 1.): behavioural statein 2.): the nervousness of the marketin 3.): the disease stage
Roland Langrock | Bielefeld University 7 / 47
Inference in HMM-type models
Why nonparametric?
1. specifying a suitable model can be hard — lots of ways to get it wrong!
2. more flexibility, perhaps leading to models that are more parsimonious,e.g. in terms of the number of states
3. as an exploratory tool
A strategy applicable in many scenarios combines
the simple yet powerful HMM machinery ...
... and the conceptual simplicity and general advantages of P-splines
Roland Langrock | Bielefeld University 8 / 47
1 Some basics on hidden Markov models
2 Nonparametric inference in hidden Markov models
3 Markov-switching generalized additive models
4 Concluding remarks
Roland Langrock | Bielefeld University 9 / 47
Some basics on hidden Markov models
Roland Langrock | Bielefeld University 10 / 47
HMMs — summary/definition
St−1 0St 0 St+1
Xt−1 0Xt 0 Xt+1
. . . . . . (hidden)
(observed)
two (discrete-time) stochastic processes, one of them hidden
distribution of observations determined by underlying state
hidden state process is an N-state Markov chain
Roland Langrock | Bielefeld University 11 / 47
Building blocks of HMMs
{St}t=1,2,...,T is (usually) assumed to be an N-state Markov chain:• state transition probabilities: γij = Pr(St = j |St−1 = i)• transition probability matrix (t.p.m.):
Γ =
γ11 . . . γ1N...
. . ....
γN1 . . . γNN
• initial state distribution: δ =
(Pr(S1 = 1), . . . ,Pr(S1 = N)
)
State-dependent distributions f (xt | st = j):• specify suitable class of parametric distributions• e.g. normal, Poisson, Bernoulli, multivariate normal, gamma, Dirichlet, ...• one set of parameters for each state
Roland Langrock | Bielefeld University 12 / 47
HMMs — likelihood calculation using brute force
L(θ) = f (x1, . . . , xT )
=N∑
s1=1
. . .
N∑sT =1
f (x1, . . . , xT , s1, . . . , sT )
=N∑
s1=1
. . .N∑
sT =1
f (x1, . . . , xT |s1, . . . , sT )f (s1, . . . , sT )
=N∑
s1=1
. . .N∑
sT =1
δs1
T∏t=1
f (xt |st )T∏
t=2
γst−1,st
Simple form, but O(TNT ), numerical maximiz. of this expression thus infeasible.
Roland Langrock | Bielefeld University 13 / 47
HMMs — likelihood calculation via forward algorithm
Consider instead the so-called forward probabilities,
αt (j) = f (x1, . . . , xt , st = j)
These can be calculated using an efficient recursive scheme:
α1 = δQ(x1)
αt = αt−1ΓQ(xt )
with Q(xt ) = diag(f (xt |st = 1), . . . , f (xt |st = N)
)
⇒ L(θ) =N∑
j=1
αT (j) = δQ(x1)ΓQ(x2) · . . . · ΓQ(xT )1
Computational effort: O(TN2) — linear in T !
Roland Langrock | Bielefeld University 14 / 47
Further inference — a brief overview
uncertainty quantification→ (parametric) bootstrap or Hessian-based
model selection→ criteria such as the AIC
model checking→ quantile residuals, simulation-based, ...
state decoding→ Viterbi algorithm
Roland Langrock | Bielefeld University 15 / 47
Related model classes
state-space models −→ can be approximated arbitrarily accurately byHMMs by finely discretizing the state space
Markov-switching regression models −→ HMMs with covariates
Markov-modulated Poisson processes −→ can be regarded as HMMs(with slightly modified dependence structure)
The corresponding likelihoods can be written as easy-to-evaluate matrix products!
Roland Langrock | Bielefeld University 16 / 47
Nonparametric inference in hidden Markov models
Roland Langrock | Bielefeld University 17 / 47
HMMs — motivation for a nonparametric approach
distribution of observations selected by underlying state
state-dependent distributions usually from a class of parametric distributions
finding the “right” distribution, or even a suitable one, can be difficult
an unfortunate choice can lead to ...• ... a poor fit and hence poor predictive power• ... a bad performance of the state decoding• ... invalid inference e.g. on the number of states
0 200 400 600 800 1000
−40
−20
020
40
observed time series
time
obse
rvat
ions
histogram of observations
observations
Fre
quen
cy
−60 −40 −20 0 20 40 60
020
4060
What family of distributions to use for the state-dependent process?
Roland Langrock | Bielefeld University 18 / 47
Nonparametric estimation based on P-splines
represent densities of state-dep. distributions using standardized B-splinebasis densities:
f (xt |st = i) =∑K
k=−K ai,kφk (xt )
transform constrained parameters ai,−K , . . . , ai,K :
ai,k =exp(βi,k )∑K
j=−K exp(βi,j )with βi,0 = 0
numerically maximize the penalized log-likelihood:
lp(θ,λ) = log(L(θ)
)−
[N∑
i=1
λi
2
K∑k=−K+2
(∆2ai,k
)2
]
Roland Langrock | Bielefeld University 19 / 47
Inference
identifiability holds under fairly weak conditions(essentially there needs to be serial correlation)
generalized cross-validation or AIC-type statistic for(i) choosing λ from N-dimensional grid(ii) model selection on the number of states
parameter estimation by numerical maximization of lp(θ,λ)
local maxima can be an issue→ use many different initial values in the maximization
uncertainty quantification via parametric bootstrap
model checking via pseudo-residuals (standard)
state decoding using Viterbi (standard)
Roland Langrock | Bielefeld University 20 / 47
A simple simulation experiment
simulate T = 800 observations from 2-state HMM
Γ =
(0.9 0.10.1 0.9
)
−60 −40 −20 0 20 40 60 80
0.00
0.01
0.02
0.03
0.04
true densities of the state−dep. distributions
dens
ity
Roland Langrock | Bielefeld University 21 / 47
A simple simulation experiment
simulate T = 800 observations from 2-state HMM
Γ =
(0.9 0.10.1 0.9
)
−60 −40 −20 0 20 40 60 80
0.00
00.
005
0.01
00.
015
0.02
0
marginal distribution of obs.
dens
ity
Roland Langrock | Bielefeld University 22 / 47
A simple simulation experiment
simulate T = 800 observations from 2-state HMM
Γ =
(0.9 0.10.1 0.9
)K = 15, thus 2K + 1 = 31 B-spline basis functions
−60 −40 −20 0 20 40 60 80
0.00
0.01
0.02
0.03
0.04
true (black) and estimated densities of the state−dep. distributions
dens
ity
lambdas about right
Roland Langrock | Bielefeld University 23 / 47
A simple simulation experiment
simulate T = 800 observations from 2-state HMM
Γ =
(0.9 0.10.1 0.9
)K = 15, thus 2K + 1 = 31 B-spline basis functions
−60 −40 −20 0 20 40 60 80
0.00
0.01
0.02
0.03
0.04
true (black) and estimated densities of the state−dep. distributions
dens
ity
lambdas too big
Roland Langrock | Bielefeld University 24 / 47
A simple simulation experiment
simulate T = 800 observations from 2-state HMM
Γ =
(0.9 0.10.1 0.9
)K = 15, thus 2K + 1 = 31 B-spline basis functions
−60 −40 −20 0 20 40 60 80
0.00
0.01
0.02
0.03
0.04
true (black) and estimated densities of the state−dep. distributions
dens
ity
lambdas too small
Roland Langrock | Bielefeld University 25 / 47
Blainville’s beaked whale dive data−
6−
4−
20
24
observed time series
time in hours
log(
|dep
th d
ispl
acem
ent|
in m
eter
s)
10 20 30 40
histogram of the observations
log(|depth displacement| in meters)
Den
sity
val
ue
−8 −6 −4 −2 0 2 4 6
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
sample ACF
lag
AC
F
Roland Langrock | Bielefeld University 26 / 47
Blainville’s beaked whale — parametric HMMs
Table: Results of fitting HMMs with normal state-dependent distributions.
#states p AIC BIC
3 12 9784.00 9855.594 20 9498.16 9617.475 30 9400.30 9579.276 42 9294.88 9545.437 56 9208.04 9542.118 72 9129.15 9558.679 90 9090.98 9627.8710 110 9064.53 9720.74
Roland Langrock | Bielefeld University 27 / 47
Blainville’s beaked whale — parametric HMM, N = 7
fitted state−dependent distributions (3−state parametric HMM)
log(absolute depth displacement)
Den
sity
−6 −4 −2 0 2 4 6
0.00
0.05
0.10
0.15
0.20
0.25
0.30
state 1state 2state 3state 4state 5state 6state 7marginal
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
qq−plot of residuals against standard normal
quantiles of the standard normal
sam
ple
quan
tiles
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
sample ACF for series of residuals
lagA
CF
Roland Langrock | Bielefeld University 28 / 47
Blainville’s beaked whale — parametric HMM, N = 3
fitted state−dependent distributions (3−state parametric HMM)
log(absolute depth displacement)
Den
sity
−6 −4 −2 0 2 4 6
0.00
0.05
0.10
0.15
0.20
0.25
0.30
state 1state 2state 3marginal
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
qq−plot of residuals against standard normal
quantiles of the standard normal
sam
ple
quan
tiles
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
sample ACF for series of residuals
lagA
CF
Roland Langrock | Bielefeld University 29 / 47
Blainville’s beaked whale — nonparametric HMM with N = 3
fitted state−dependent distributions (3−state nonparametric HMM)
log(absolute depth displacement)
Den
sity
−6 −4 −2 0 2 4 6
0.00
0.05
0.10
0.15
0.20
0.25
0.30
state 1state 2state 3marginal
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
qq−plot of residuals against standard normal
quantiles of the standard normal
sam
ple
quan
tiles
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
sample ACF for series of residuals
lagA
CF
Roland Langrock | Bielefeld University 30 / 47
Blainville’s beaked whale — Viterbi for nonparametric HMM with N = 3
time in hours
−4
−2
0
2
4
2 4 6 8
state 3
state 2
state 1
1200
1000
800
600
400
200
0
● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●
●●●●●●●●●● ●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●● ●● ●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●
●●●●●●●●●●●● ●●●● ● ●●●●●●●●●●● ●●●●●●●●●●● ●●●●●●●●●●●●●●●● ●●●●●●●● ●●● ●●●● ●●●●●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●
d
epth
s in
met
ers
l
og(|
dept
h di
spla
cem
ent|
in m
eter
s) d
ecod
ed s
tate
s
Roland Langrock | Bielefeld University 31 / 47
Markov-switching generalized additive models
Roland Langrock | Bielefeld University 32 / 47
Markov-switching regression — a basic model
A simple Markov-switching (linear) regression model:
Yt = β(st )0 + β
(st )1 xt + σst εt ,
with
a time series {Yt}t=1,...,T
associated covariates x1, . . . , xT (including the possibility of xt = yt−1)
εtiid∼ N (0, 1)
st : state at time t of an unobservable N-state Markov chain
Roland Langrock | Bielefeld University 33 / 47
Markov-switching regression — remarks on the basic model
commonly used in economics to deal with parameter instability over time(key references: Goldfeld and Quandt, 1973; Hamilton, 1989)
linear form of the predictor is usually assumed with little investigation (if any!)into the absolute or relative goodness of fit
we consider nonparametric methods for estimating the form of the predictor(in analogy to the extension of linear models to GAMs)
Roland Langrock | Bielefeld University 34 / 47
Markov-switching regression — more general model formulation
More general model formulation:
g(E(Yt | st , x·t )︸ ︷︷ ︸
µ(st )t
)= η(st )(x·t ),
where
Yt follows some distribution from the exponential family
x·t = (x1t , . . . , xPt ) is the covariate vector at time t
g is a suitable link function
η(st ) is the predictor function given state st
(the form of which we do not yet specify)
(φ(st ): any additional state-dependent dispersion parameters)
Roland Langrock | Bielefeld University 35 / 47
Likelihood evaluation using the forward recursion
Define, analogously as for HMMs, the forward variable
αt (j) = f (y1, . . . , yt ,St = j | x·1 . . . x·t )
Then the following recursive scheme can be applied:
α1 = δQ(y1) , αt = αt−1ΓQ(yt ) (t = 2, . . . ,T )
whereQ(yt ) = diag
(pY (yt ;µ
(1)t , φ(1)), . . . , pY (yt ;µ
(N)t , φ(N))
)
⇒ L(θ) =N∑
j=1
αT (j) = δQ(x1)ΓQ(x2) · . . . · ΓQ(xT )1
This form applies for any form of the conditional density pY (yt ;µ(st )t , φ(st ))
Roland Langrock | Bielefeld University 36 / 47
Nonparametric modelling of the predictor
here we consider a GAM-type framework:
η(st )(x·t ) = β(st )0 + f (st )
1 (x1t ) + f (st )2 (x2t ) + . . .+ f (st )
P (xPt ),
we represent each f (i)p as a linear combination of B-spline basis functions:
f (i)p (x) =K∑
k=1
γipk Bk (x)
... and numerically maximize the penalized log-likelihood:
lp(θ,λ) = log(L(θ)
)−
N∑i=1
P∑p=1
λip
2
K∑k=3
(∆2γipk )2
inference analogous as for nonparametric HMMs
notably, parametric models are nested special cases (for λ→∞)
Roland Langrock | Bielefeld University 37 / 47
A simple simulation experiment
simulate T = 300 observations from 2-state Markov-switching regr. model:
Yt ∼ Poisson(eβ0+f (st )(xt )), Γ =
(0.9 0.10.1 0.9
)
−3 −2 −1 0 1 2 3
−6
−4
−2
02
46
xt
f(st) (x
t)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
Roland Langrock | Bielefeld University 38 / 47
A simple simulation experiment
simulate T = 300 observations from 2-state Markov-switching regr. model:
Yt ∼ Poisson(eβ0+f (st )(xt )), Γ =
(0.9 0.10.1 0.9
)K = 15, thus 2K + 1 = 31 B-spline basis functions
smoothing parameter selection from a grid using AIC-type statistic
−3 −2 −1 0 1 2 3
−6
−4
−2
02
46
xt
f(st) (x
t)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
Roland Langrock | Bielefeld University 39 / 47
A simple simulation experiment
simulate T = 300 observations from 2-state Markov-switching regr. model:
Yt ∼ Poisson(eβ0+f (st )(xt )), Γ =
(0.6 0.40.4 0.6
)K = 15, thus 2K + 1 = 31 B-spline basis functions
smoothing parameter selection from a grid using AIC-type statistic
−3 −2 −1 0 1 2 3
−6
−4
−2
02
46
xt
f(st) (x
t)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
st=1 (state 1)
st=2 (state 2)
Roland Langrock | Bielefeld University 40 / 47
Example Lydia Pinkham sales1.
01.
52.
02.
53.
03.
5
year
sale
s (in
mill
ion
US
D)
1910 1920 1930 1940 1950 1960
Roland Langrock | Bielefeld University 41 / 47
Example Lydia Pinkham sales
Model MS-LIN: salest = β(st )0 + β
(st )1 advertisingt + β
(st )2 salest−1 + σst εt
Model MS-GAM: salest = β(st )0 + f (st )(advertisingt ) + β
(st )1 salest−1 + σst εt
0.5 1.0 1.5 2.0
1.5
2.0
2.5
3.0
MS−LIN
Advertising
Sal
es
0.5 1.0 1.5 2.0
1.5
2.0
2.5
3.0
MS−GAM
Advertising
Sal
es
Figure: Estimated state-dependent mean sales as functions of advertising expenditure(state 1 in green, state 2 in red). Displayed are the predictor values when fixing theregressor salest−1 at its overall mean, 1.84.
Roland Langrock | Bielefeld University 42 / 47
Example Lydia Pinkham sales
● ●●
●
●
●
● ●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
● ●
●● ●
●
● ●
●
1910 1920 1930 1940 1950 1960
1.0
1.5
2.0
2.5
3.0
3.5
year
sale
s
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
1910 1920 1930 1940 1950 1960
year
stat
e
12
Figure: Sales figures and decoded states underlying the MS-GAM model.
Roland Langrock | Bielefeld University 43 / 47
Example Lydia Pinkham sales
0 5 10 15
−0.
20.
20.
61.
0
Lag
AC
F
ACF MS−LIN residuals
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
qq−plot MS−LIN residuals
Theoretical Quantiles
Sam
ple
Qua
ntile
s
0 5 10 15
−0.
20.
20.
61.
0
Lag
AC
F
ACF MS−GAM residuals
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
qq−plot MS−GAM residuals
Theoretical Quantiles
Sam
ple
Qua
ntile
s
Roland Langrock | Bielefeld University 44 / 47
Concluding remarks
Roland Langrock | Bielefeld University 45 / 47
Concluding remarks
bringing together HMMs & P-splines gives lots of modelling options
while inference is slightly more involved, resulting models often substantiallyincrease the goodness of fit, and may in fact be more parsimonious thanparametric alternatives
various other such models can be formulated (and fitted), e.g. MS-GAMLSSmodels — but does anyone need this kind of thing??
we’re currently working on alternative, less computer-intensive methods forselecting the smoothing parameters
Roland Langrock | Bielefeld University 46 / 47
References
Langrock, R., Kneib, T., Sohn, A., DeRuiter, S. (2015), “Nonparametricinference in hidden Markov models using P-splines”, Biometrics
Langrock, R., Glennie, R., Kneib, T., Michelot, T. (2016). “Markov-switchinggeneralized additive models”, Statistics and Computing
Thank you!
Roland Langrock | Bielefeld University 47 / 47
Recommended