Lecture 4, Non-linear Time SeriesMagnus Wiktorsson
“It’s not a bug, it’s a feature!”
▶ Why are we using linear models?▶ Properties▶ Limitations
▶ Properties of non-linear systems.▶ Limit cycles▶ Jumps▶ Non-symmetric distributions▶ Bifurcations▶ Chaos▶ Non-linear dependence
“It’s not a bug, it’s a feature!”
▶ Why are we using linear models?▶ Properties▶ Limitations
▶ Properties of non-linear systems.▶ Limit cycles▶ Jumps▶ Non-symmetric distributions▶ Bifurcations▶ Chaos▶ Non-linear dependence
General properties
▶ Assume causal system
f(Yn,Yn−1, . . . ,Y1) = εn
▶ Invertable system
Yn = f⋆(εn, . . . , ε1)
▶ Volterra series.
Suppose that f∗ is sufficientlywell-behaved, then there exists a sequence of boundedfunctions∞∑
k=0|ψk| <∞,
∞∑k=0
∞∑l=0
|ψkl| <∞,∞∑
k=0
∞∑l=0
∞∑m=0
|ψklm| <∞, ...
General properties
▶ Assume causal system
f(Yn,Yn−1, . . . ,Y1) = εn
▶ Invertable system
Yn = f⋆(εn, . . . , ε1)
▶ Volterra series. Suppose that f∗ is sufficientlywell-behaved, then there exists a sequence of boundedfunctions∞∑
k=0|ψk| <∞,
∞∑k=0
∞∑l=0
|ψkl| <∞,
∞∑k=0
∞∑l=0
∞∑m=0
|ψklm| <∞, ...
Volterra serieswhere
µ = f∗(0), ψk = (∂f∗∂ϵt−k
), ψkl = (∂2f∗
∂ϵt−k∂ϵt−l), ... (1)
Approximate the general model by
Yt = µ+∞∑
k=0ψkϵt−k +
∞∑k=0
∞∑l=0
ψklϵt−kϵt−l
+∞∑
k=0
∞∑l=0
∞∑m=0
ψklmϵt−kϵt−lϵt−m + . . . (2)
This results in generalized transfer functions. NOTE thatsuperposition is lost!
These transfer functions do not care if {ϵ} is deterministicof stochastic!
Volterra serieswhere
µ = f∗(0), ψk = (∂f∗∂ϵt−k
), ψkl = (∂2f∗
∂ϵt−k∂ϵt−l), ... (1)
Approximate the general model by
Yt = µ+∞∑
k=0ψkϵt−k +
∞∑k=0
∞∑l=0
ψklϵt−kϵt−l
+∞∑
k=0
∞∑l=0
∞∑m=0
ψklmϵt−kϵt−lϵt−m + . . . (2)
This results in generalized transfer functions.
NOTE thatsuperposition is lost!
These transfer functions do not care if {ϵ} is deterministicof stochastic!
Volterra serieswhere
µ = f∗(0), ψk = (∂f∗∂ϵt−k
), ψkl = (∂2f∗
∂ϵt−k∂ϵt−l), ... (1)
Approximate the general model by
Yt = µ+∞∑
k=0ψkϵt−k +
∞∑k=0
∞∑l=0
ψklϵt−kϵt−l
+∞∑
k=0
∞∑l=0
∞∑m=0
ψklmϵt−kϵt−lϵt−m + . . . (2)
This results in generalized transfer functions. NOTE thatsuperposition is lost!
These transfer functions do not care if {ϵ} is deterministicof stochastic!
Frequency doubling
Now assume that we introduce a spectral representation ofthe noise.▶ Let’s start with a single frequency, ϵk = A exp(iω ∗ k)
▶ This results in frequency doubling▶ Proof by inserting the signal in Eq (2).▶ Question: What happens with a non-linear system if
the noise ϵk is white noise?▶ Conclusion: Black box non-linear system identification
is far more complicated that linear systemidentification.
Frequency doubling
Now assume that we introduce a spectral representation ofthe noise.▶ Let’s start with a single frequency, ϵk = A exp(iω ∗ k)▶ This results in frequency doubling▶ Proof by inserting the signal in Eq (2).
▶ Question: What happens with a non-linear system ifthe noise ϵk is white noise?
▶ Conclusion: Black box non-linear system identificationis far more complicated that linear systemidentification.
Frequency doubling
Now assume that we introduce a spectral representation ofthe noise.▶ Let’s start with a single frequency, ϵk = A exp(iω ∗ k)▶ This results in frequency doubling▶ Proof by inserting the signal in Eq (2).▶ Question: What happens with a non-linear system if
the noise ϵk is white noise?
▶ Conclusion: Black box non-linear system identificationis far more complicated that linear systemidentification.
Frequency doubling
Now assume that we introduce a spectral representation ofthe noise.▶ Let’s start with a single frequency, ϵk = A exp(iω ∗ k)▶ This results in frequency doubling▶ Proof by inserting the signal in Eq (2).▶ Question: What happens with a non-linear system if
the noise ϵk is white noise?▶ Conclusion: Black box non-linear system identification
is far more complicated that linear systemidentification.
Regime models
The model is generated from a set of simple models▶ SETAR▶ STAR▶ HMM
SETAR - Self-Exciting Threshold AR
The SETAR(l; d; k1, k2, . . . , kl) model is given by :
Yt = a(Jt)0 +
kJt∑i=1
a(Jt)i Yt−i + ϵ
(Jt)t (3)
where the index (Jt) is described by
Jt =
1 for Yt−d ∈ R12 for Yt−d ∈ R2... ...l for Yt−d ∈ Rl.
(4)
NOTE that it is difficult to estimate the boundaries for theregimes
SETAR - Self-Exciting Threshold AR
The SETAR(l; d; k1, k2, . . . , kl) model is given by :
Yt = a(Jt)0 +
kJt∑i=1
a(Jt)i Yt−i + ϵ
(Jt)t (3)
where the index (Jt) is described by
Jt =
1 for Yt−d ∈ R12 for Yt−d ∈ R2... ...l for Yt−d ∈ Rl.
(4)
NOTE that it is difficult to estimate the boundaries for theregimes
SETARMA
▶ Similar ideas can be included in ARMA models,leading to SETARMA models.
▶ Often easy to add ’asymmetric’ terms in the AR orMA polynomials, e.g.
yn + a1yn−1 = en +(c1 + c′11{en−1≤0}
)en−1
STAR - Smooth Threshold ARThe STAR(k) model:
Yt = a0+k∑
j=1ajYt−j+
b0 +k∑
j=1bjYt−j
G(Yt−d)+ϵt (5)
where G(Yt−d) now is the transition function lying betweenzero and one, as for instance the standard Gaussiandistribution.In the literature two specifications for G(·) are commonlyconsidered, namely the logistic and exponential functions:
G(y) = (1 + exp(−γL(y − cL)))−1; γL > 0 (6)
G(y) = 1 − exp(−γE(y − cE)2); γE > 0 (7)
where γL and γE are transition parameters, cL and cE arethreshold parameters (location parameters).
PJM electricity market
Prices at the PJM market
Simple model of the power market
▶ Demand
D(Q) = a + bQ + c cos(2πt/50) + ε (8)
▶ Supply
S(Q) = α0+β0Q+G(Q,Qbreak)(α1+β1(Q−Qbreak)+)(9)
where G is a transition function.▶ Solve numerically for t = 1, . . . to get the quantity Q
and price P.
Supply and Demand
50 60 70 80 90 100 110 1200
100
200
300
400
500
600
700
800
900
1000
Supply
MaxDemand
MinDemand
Figure: Supply and demand curves (varies across the season) forour artificial market
Prices
0 50 100 150 200 250
250
300
350
400
450
500
550
600
650
700
750
Figure: Note the seasonality as well as the non-Gaussiandistribution.
Distribution of prices
300 350 400 450 500 550 600 650 700
Data
0.001
0.003
0.010.02
0.05
0.10
0.25
0.50
0.75
0.90
0.95
0.980.99
0.997
0.999
Pro
ba
bili
ty
Normal Probability Plot
Figure: Same property
HMM - Hidden Markov Models
Another alternative is to let the regime shift stochastically,as in the Hidden Markov Model. Let
Yt = a(Jt)0 +
kJt∑i=1
a(Jt)i Yt−i + ϵ
(Jt)t (10)
where the state variable Jt follows a latent Markov chain.
NOTE that parameter estimation is slightly morecomplicated than before.
HMM - Hidden Markov Models
Another alternative is to let the regime shift stochastically,as in the Hidden Markov Model. Let
Yt = a(Jt)0 +
kJt∑i=1
a(Jt)i Yt−i + ϵ
(Jt)t (10)
where the state variable Jt follows a latent Markov chain.
NOTE that parameter estimation is slightly morecomplicated than before.
Case: Electricity spot price, (Regland &Lindström, 2012)
The electricity spot price is very non-Gaussian
Feb05 Feb07 Feb09
0
50
100
150
200
250
EEX spot
Feb05 Feb07 Feb09−4
−3
−2
−1
0
1
EEX log(spot)−log(forward)
Figure: The electricity spot price (left) and spread, defined as thedifference between the logarithm of the spot and the logarithmof the forward (right). Data from the German EEX market.
▶ The spread accounts for virtually all seasonality, butthere are still bursts of volatility.
▶ The logarithm of the spot, yt, was modeled using aHMM regime switching model with three states, anormal state with mean-reverting dynamics, a spike(upward jumps) state and a drop (downward jumps)state.
This is mathematically given by :
∆y(B)t+1 = α
(µt − y(B)
t
)+ σϵt
y(S)t+1 = ZS,t + µt, ZS ∼ F (µS, σS)
y(D)t+1 = −ZD,t + µt, ZD ∼ F (µD, σD)
where µt is approximately the logarithm of the monthahead forward price.The regimes are switching according to a Markov chainRt = {B,S,D} governed by the transition matrix
Π =
1 − πBU − πBD πBS πBDπSB 1 − πSB 0πDB 0 1 − πDB
.
Feb05 Feb06 Feb07 Feb08 Feb09 Feb10−4
−3
−2
−1
0
1
Spr
ead
Feb05 Feb06 Feb07 Feb08 Feb09 Feb10−1
0
1
Reg
ime
prob
Figure: Fit of the independent spike model applied to EEX data
Extension used for stability evaluation of the power systemin (Lindström, Norén & Madsen, 2015) by making thetransition matrix time-inhomogeneous.
Case: What happens with large scaleintroduction of electric cars/battery?
Jan02 Jan04 Jan06 Jan08 Jan10 Jan12
0.4
0.5
0.6
0.7
0.8
0.9
1
Mod
ified
Nor
mal
ized
Con
sum
ptio
n
0 %10 %
Battery capacity (%) 0 5 10 15Base prob. 0.8794 0.8827 0.9066 0.9461Spike prob. 0.0304 0.0292 0.0196 0.0081Drop prob. 0.0902 0.0881 0.0738 0.0458
Table: Unconditional regime probabilities when having a perfectbattery with 0, 5, 10, and 15 % system capacity.
HMMs for portfolio optimizationRecall the stylized facts for stock indices.
We can also useHMMs for portfolio optimization.▶ Model given by
Xt = µSt + εSt (11)
with µ1, µ2, σ1, σ2, π1 stat. prob. for the first stateand λ = γ11 + γ22 − 1 is the second largest eigenvalueto the transition matrix, Γ.
First, consider the autocorrelation (for k > 0):
r(k) = π1(1 − π1)(µ1 − µ2)2
σ21π1 + σ2
2(1 − π1) + π1(1 − π1)(µ1 − µ2)2λk
(12)See Nystrup et al (2016) for details.
HMMs for portfolio optimizationRecall the stylized facts for stock indices. We can also useHMMs for portfolio optimization.▶ Model given by
Xt = µSt + εSt (11)
with µ1, µ2, σ1, σ2, π1 stat. prob. for the first stateand λ = γ11 + γ22 − 1 is the second largest eigenvalueto the transition matrix, Γ.
First, consider the autocorrelation (for k > 0):
r(k) = π1(1 − π1)(µ1 − µ2)2
σ21π1 + σ2
2(1 − π1) + π1(1 − π1)(µ1 − µ2)2λk
(12)See Nystrup et al (2016) for details.
Simulation example
Here the model is given by
Γ =
[0.98 0.020.1 0.9
].
with µ = [0.01 − 0.02] and σ = [0.04 0.20].Interpretation of parameters: Staying on average1/(1 − 0.98) = 50 days in the good state vs 10 days in thebad state.
Realizations
100 200 300 400 500 600 700 800 900 1000
0
2
4
6
0 100 200 300 400 500 600 700 800 900 1000
1
1.5
2
0 100 200 300 400 500 600 700 800 900 1000
-1
0
1
Figure: Cumulative returns (top), Markov states (middle) andreturns (bottom).
Autocorrelation
0 2 4 6 8 10 12 14 16 18 20
Lag
-0.2
0
0.2
0.4
0.6
0.8
1
Sam
ple
Auto
corr
ela
tion
Sample Autocorrelation Function
0 2 4 6 8 10 12 14 16 18 20
Lag
-0.2
0
0.2
0.4
0.6
0.8
1
Sam
ple
Auto
corr
ela
tion
Sample Autocorrelation Function
Figure: Autocorrelation for returns (left) and abs returns (right)
Trading strategy
Figure: Trading strategy in US stocks and bonds
General State space models
A Grey box approach is to include as much prior knowledgeas possible.
Consider the General State Space model:
xn+1 = f(n, xn, un) + g(n, xn, un)en+1
yn+1 = h(n + 1, xn+1, un+1) + wn+1
where {xn}n≥0 is a latent process and {yn}n≥0 is thesequence of observations.▶ Interpretations?▶ Practical considerations
General State space models
A Grey box approach is to include as much prior knowledgeas possible.
Consider the General State Space model:
xn+1 = f(n, xn, un) + g(n, xn, un)en+1
yn+1 = h(n + 1, xn+1, un+1) + wn+1
where {xn}n≥0 is a latent process and {yn}n≥0 is thesequence of observations.▶ Interpretations?▶ Practical considerations
Example: The Black & Scholes model
The Black & Scholes (1973) model is often used for optionvaluation.
x :dS = µStdt + σStdWt,
y :
[SMarket
ncMarket
K (Sn, ·)
]=
[SModel
ncModel
K (Sn, ·)
]+ wn.︸︷︷︸
Ask-Bix spread
This structure allows us to separate actual price variationfrom market micro structure.
Some references
▶ Lindström, E., & Regland, F. (2012). Modelingextreme dependence between European electricitymarkets. Energy economics, 34(4), 899-904. http://dx.doi.org/10.1016/j.eneco.2012.04.006
▶ Lindström, E., Norén, V., & Madsen, H. (2015).Consumption management in the Nord Pool region: Astability analysis. Applied Energy, 146, 239-246.http://dx.doi.org/10.1016/j.apenergy.2015.01.113
▶ Nystrup, P., Hansen, B. W., Madsen, H., &Lindström, E. (2015). Regime-based versus staticasset allocation: Letting the data speak. The Journalof Portfolio Management, 42(1), 103-109.http://dx.doi.org/10.3905/jpm.2015.42.1.103
Cont.▶ Nystrup, P., Madsen, H., & Lindström, E. (2016). Long
memory of financial time series and hidden Markov modelswith time-varying parameters. Journal of Forecastinghttp://dx.doi.org/10.1002/for.2447
▶ Lindström, E., Ströjby, J., Brodén, M., Wiktorsson, M., &Holst, J. (2008). Sequential calibration of options.Computational Statistics & Data Analysis, 52(6),2877-2891.
▶ Lindström, E., & Gou, J. (2013). Simultaneous calibrationand quadratic hedging of options. Quantitative andQualitative Analysis in Social Sciences
▶ Lindström, E., & Åkerlindh, C. (2018). Optimal AdaptiveSequential Calibration of Option Models. In Handbook ofRecent Advances in Commodity and Financial Modeling(pp. 165-181). Springer,https://doi.org/10.1007/978-3-319-61320-8_8