Embed Size (px)
Citation preview
Connecting Nonlinear Time Series Analysis with Physics
Henry D. I. Abarbanel
Department of Physics
and
Marine Physical Laboratory (Scripps Institution of Oceanography)
University of California, San Diego
Outline of this talk:
I received, via email, a time series of light output from variable stars.
This put me into a situation that is a bit awkward for a Physicist who wants to learn about the processes which produced the data and doesn’t know the details about the observations and the instruments used to make them.
I do not know the Physics that underlies these data
I came here to find out, actually.
I do not know the literature on analyzing these time series or anything about the models one uses to describe variable stars.
So, violating my moderately strongly held principles about never working with data about which I know nothing, I put on an old, worn engineering hat, and went to work to see if some constraints on any relevant Physics, expressed in a model, could be extracted from the data sent along.
===================
Here are some observations about the data I received:
The time series has many gaps, and I rejected any tactic to fill in those gaps. I selected one continuous segment of the data from 975 time units to 1000 time units. Δt = 0.2043 (somethings). The minimum in the data is -1.34 and the maximum is 1.57 (not π/2). The mean value is -0.002, so zero. The data comprises 1211 data points, and it looks like this:
Outline of this talk:
It looks close to periodic, but not quite. One would expect there to be a few sharp frequencies in its Fourier spectrum (a linear operation on signals from nonlinear oscillators—but of some use). The “peaks” in the power spectrum should be accompanied by a broadband background.
SO: we do the following actions—explanations to accompany the results.
examine the Fourier power spectrum;
move promptly back to time domain for any further discussion;
ask about nonlinear correlations among parts of the data. This is the average mutual information as a function of time lag;
select a time lag T where the average mutual information has a first minimum;
build from this a “proxy” state space which undoes the projections from the multidimensional phase space where the star’s dynamics happens to the observation axis.
The observed light intensity as a function of time I(t), a scalar, is projected into a signal vector in a DE dimensional space:
how shall we choose DE and T ? DE is an integer.
Working in this space, where the orbit S(t) is not self-intersecting, evaluate characteristics of the signal on its attractor. Explanations accompany the results.
Percentage of global false nearest neighbors as a function of DE: 0 is desired.
Integer dimension where dynamics operates—local false nearest neighbors
examine the attractor of the dynamics in DE (well, three) dimensional space
Lyapunov exponents—index of stability of the underlying nonlinear processes; Lyapunov dimension (not integer).
predict ? Can only predict light intensity for the same forcing conditions of the physical system. Rather limited situation.
When we have done all this we will know a small number of characteristics of the processes which produced the signal.
These are constraints on the output of any model we make of variable stars.
This is not the core Physics.
===========
Time to toss the engineering hat and put on the sturdy “get the Physics” hat!
Average mutual information acts as a kind of nonlinear correlation function relating the signal now to the signal T time steps later—on the average over the signal:
3T
False Nearest Neighbors
In projecting from the dimension D of the variable star dynamics down to I(t),we create phase space points which are neighbors by projection, not by dynamical processes.
By “unprojecting” we remove these false neighbors so we have a chance atestablishing a dynamical rule of the form
This connects locations S(tn) in that space to real dynamical neighbors S(tn+1) in that space at the next time step.
1
E
( ) ( ) ( ( ))
in D dimensional spacen n nS t S t f S t
5 7 ?ED
Local False Nearest Neighbors
The global quantity false nearest neighbors tells us the necessary phase space dimension where points are separated by dynamics.
Local false nearest neighbors tells us a lower or equal phase space dimension where the local dynamical motion occurs.
Useful example is motion on a Moebius strip: global dimension is three; local dimension, where dynamics occurs, is two.
local 4D
Lyapunov Exponents Stability Coefficients
local Lyapunov5 4 3.89ED D D
What do we know so far?
(a) The signal comes from the operation of some differential equations. (b) The orbits lie on a strange attractor of dimension about 3.9. (c) The dynamics have 5 or 6 dynamical variables. (d) Errors grow as
By watching where a new observation is located in S(t) space, we can look at its nearest neighbors and predict where it will be at S(t+T).
===================
So far we have no knowledge of any of the actual physical state variables except I(t).
[ ( )/2.6]( ) (0) t unitsI t I e
Move from engineering to Physics.
Use information in measurements to inform a model of the Physical processes about any unknown fixed parameters in the model and about any unobserved state variable in the model.
Observe y(t) over [t0,tf]. Estimate parameters and all unobserved states over this interval. From full state x(tf) and p---predict for t > tf.
Measurements are noisy
Model has errors
The problem is intrinsically stochastic
What we must evaluate is the joint probability distribution of the states in the observation window conditioned on the measurements:
0 1, ,... ,...,n mt t t t t T
( )
1, ... ,l ny t
l L
1( )
( ( ));
1,2,...,
a n
a n
x t
f x t
a D
( ) ( ( ))l ly n h x n
L D
Data source: Transmitter
Model: Receiver
Using Bayes rule and the Markov property of
the dynamics, w recursion relatione have the :
( ( ) | ( exp[ ( ( ), ( ) | ( -1))]
( 1) ( ( ) | ( 1))
))
( ( -1) | ( -1))Dx
P x m Y m
P
CMI x m y m Y m
d m P x m x m Y mm x
Probability of x(m) after m+1 measurements
Probability of x(m-1) after m measurements
( ( ), ( ) | ( -1))
( ( ) | ( -1)) ( ( ) | ( -1))( ( ), ( ) | ( -1)) log
P x m y m Y m
P y m Y m P x m Y mCMI x m y m Y m
0( , ( ))
1( , ( ))
01
0
=
( ), ( 1),..., (0)
( ( ) | ( ))
( ) ( ( 1) | ( )) ( (0))
( ) A X Y m
mTMI X Y mD
nm
n
D
x
x X x m x m x
T
P x m Y m
d n e P x n x n P x
d n e
m 0 P Iterate from t back to t to give ath Integrah lt e :
0( , ( )) ( ( ), ( ) | ( 1))
Total Conditional Mutual Information between path and observations ( )
m
nMI X Y m CMI y n x n Y n
X Y m
What is different about this path integral: nonlinear “propagators” dissipative dynamics, orbits on strange attractors.
Path
0
00
1
00
1
0
0- l
og{ ( (0))}
term
( | ) exp[ ( )]
( ) log{ ( ( ), ( ) | ( 1))}
- log{ ( ( 1) | ( ))} log{ ( (
( ) log{ ( ( ) | ( ), ( 1))}
- log{ ( (
0)
1) ( ))}
)}
|
m
n
nm
nm
m
nP x
A
P X Y A X
A X CMI x n y n Y n
P x n x n
X P y n x n Y n
P x n x n
P x
s independent of X
Action for State and Parameter Estimation
0
0
0
- ( )
- ( )
With the density of paths exp[- ( )],
we are able to evaluate the conditional expectation
value of a function G(X) on the path
( ), ( -1),..., (1), (0)
( ) [ ( ) | ] ( )
A X
A X
A X
X x m x m x x
dXe G XE G X Y G X
dXe
Some important G(X): X, the expected path <X>; moments about <X>;
marginal distributions G(X) = δ(U-xb(tk)) = P(U)
Laplace Method for Integrals
Variational Methods
0
0
- ( )
- ( )
( ) [ ( ) | ] ( )
A X
A X
dXe G XE G X Y G X
dXe
Variational principles. Laplace’s method (1774):
Find Xq which satisfy “saddle path” via unconstrained variation
Expand integral around the Xq to approximate the integral.
Finding the Xq is called 4DVar in geosciences.
In the context of the path integral, this has a precise form, and one can evaluate corrections to the Laplace approximation.
0( )0
qX X
A X
X
0
( ( ) | ( )) ( ( ) | ( ))( ( ) | ( )) ( ( ) | ( 1)) ( ( ) | ( 1))
( ( )) ( ( ))
If the errors in the observations are independent, then
and ( ) ( ( ), ( )) log[ ( ( 1) | ( ))] log ( (0))n
P x m y m P y m x mP x m Y m P x m Y m P x m Y m
P x m P y m
A X CMI x n y n P x n x n P x
With usual assumptions: the measurements are contaminated by Gaussian noise, …
, 1,2 2
00, 1 0, 1
Measurement Error (Synchron
(
i
)
z
( , ) ( ) [( ( ) ( ))] + (
ation) Model Er
( 1) ( ( )
ror (Finite Resolut
))2 2
ion)
log( ( (0))
m L m Dfm
l l a an l n a
R aR n lA X x n y n x n f x n
P x
Initial Condition
Approximating the Action
22 /2
2
( ( 1) | ( )) = ( ( 1) ( ( )))
( ( 1) ( ( ))(2 ) exp[ ]
2
D
D
P x n x n x n f x n
x n f x n
Finite Model Resolution
Gaussian Error Action
For the Gaussian Error Action, the corrections to the minimum action path behave as powers of 1/Rf for large Rf.
We do not yet have an argument for other noise models.
A direct attempt to find the Xq with Rm fixed by measurement error and Rf fixed by accuracy of the model—well, fails, almost surely.
In path space the multiple minima are located in steep valleys in a high dimensional space.
For an assimilation window of order T, the “size” of the minimum in path space goes as exp[-λmax T]
If we choose the dynamics of the model to be totally unresolved in x(t) space, Rf = 0, and the minimum is at xl(t) = yl(t).
This is enormously degenerate as the unmeasured states are not known.
Gaussian Error Action
An annealing strategy:a. Start with Rf = 0. Use xl(t) = yl(t) for measured states; use NO
different choices for the unmeasured states drawn from a uniform distribution across the dynamical range of the unmeasured variables. This gives NO initial paths.
b. Use these initial paths with Rf = Rf0 ≈ 0.01-0.001. Utilize your chosen numerical optimization protocol (we have used IPOPT, BFGS, and others) to find NO new paths .
c. Increase Rf to Rf = Rf0 and use paths from previous optimization to arrive at NO paths .
d. Continue increasing Rf until large. Plot NO action levels as a function of Rf .
0qX
1qX
2 ; =1 2qX
2 0log /f fR R
0( )qA X
We want to find the saddle paths Xq as a function of Rf and also find the dependence of the action levels A0(Xq) on the number of measurements at each observation time.
, 1,2 2
00, 1 0, 1
( )( , )( ) [( ( ) ( ))] + ( ( 1) ( ( )))
2 2
m L m Dfm
l l a an l n a
R aR n lA X x n y n x n f x n
As Rf becomes large, it drives the dynamics to higher accuracy, x(n+1) f(x(n)), and the action becomes weakly dependent on Rf .
When this happens the action level is dictated by
is Gaussian, by choice, and the action term is distributed as with L(m+1) degrees of freedom. This sets an expected level for the action.
,2
0, 1
( , )[( ( ) ( ))]
2
m L
ml l
n l
R n lx n y n
( ) ( )l lx n y n 2
1 1 0 1 1
1 1 2
Lorenz96 Model
Dynamical variables--longitudinal `activity'--no vertical levels
no latitude variations.
y ( ) 1, 2,...,
( ) ( ) ( ) ( ) y ( ) ( )
( )( )( ( ) ( ))
a
D D D
aa a a
t a D
y t y t y t y t t y t
dy ty t y t y t y
dt
( ) Forcinga t
We explored D = 5. Chaotic at Forcing = 7.9; we used Forcing = 8.17
15 2 4 1
21 3 5 2
32 4 1 3
43 5 2 4
24
1 1 1
2 3
1
3
( )( )( ( ) ( )) ( )
( )( )( ( ) ( )) ( )
( )( )( ( ) ( )) ( )
( )( )( ( ) ( )) ( )
(
( )( ( ) ( ))
( )( ( ) ( )
(
)
))( ( )
u t x t y t
u t
dy ty t y t y t y t F
dtdy t
y t y t y t y t Fdtdy t
y t y t y t y t Fdtdy t
x t
y t y t y t y t Fdtdy t
y t
y t
y t ydt
3 5( )) ( )t y t F
Model does not synchronize with ‘data’ (x(t)) with only one coupling: two positive Conditional Lyapunov Exponents two different couplings for data are required
NO = 100
Expected level2
NO = 100
Expected level2
NO = 100
Expected level2
NO = 100
Summary and Wrap-up
Using time series of data, working in time domain, one can determine characteristics of the source as the state x(t) moves on the system attractor. Assumes the system is stationary during the observations.
We used light intensity from variable stars to estimate that (a) The signal comes from the operation of some differential
equations. (b) The orbits lie on a strange attractor of dimension about 3.9. (c) The dynamics have 5 or 6 dynamical variables. (d) Errors grow as
By watching where a new observation is located in S(t) space, we can look at its nearest neighbors and predict where it will be at S(t+T). However, without a model connecting I(t) to other dynamical properties of the variable star, we cannot predict anything else.
[ ( )/2.6]( ) (0) t unitsI t I e
Summary and Wrap-up
Using the path integral formulation, estimate the conditional expected value of functions along the path. Expectation is conditioned on measurements.
Use Laplace approximation or Monte-Carlo approximation of the high dimensional integrals.
Require knowledge of the measurements and their relation to the state variables x(t) of the Physics in the model.
Summary and Wrap-up
Good “fits” to the observations are unrevealing about the unobserved state variables and the unknown parameters.
One critical question is this: how many measurements must be made at each observation time? This is closely related to the positive conditional Lyapunov exponents which control stability.
Prediction, deterministic or stochastic, is the key validation of the model. Response to new forcing validates the Physics.
TABLE I: Known and estimated parameters for the NaKL model. We also display the bounds used for the nonlinear search algorithm.Parameters
Known Estimated Search Lower Bound Search Upper BoundgNa 120.0 108.4 50.0 200.0ENa 50.0 49.98 0.0 100.0gK 20.0 21.11 5.0 40.0EK -77.0 -77.09 -100.0 -50.0gL 0.3 0.3028 0.1 1.0EL -54.0 -54.05 -60.0 -50.0C 0.8 0.81 0.5 1.5Vm -40.0 -40.24 -60.0 -30.0ΔVm 0.0667 0.0669 0.01 0.1m0 0.1 0.0949 0.05 0.25m1 0.4 0.4120 0.1 1.0Vh -60.0 -59.43 -70.0 -40.0ΔVh -0.0667 -0.0702 -0.1 -0.01h0 1.0 1.0321 0.1 5.0h1 7.0 7.76 1.0 15.0Vn -55.0 -54.52 -70.0 -40.0ΔVn 0.0333 0.0328 0.01 0.1n0 1.0 1.06 0.1 5.0n1 5.0 4.97 2.0 12.0
0
( ( ) | ( )) ( ( ) | ( ))( ( ) | ( )) ( ( ) | ( 1)) ( ( ) | ( 1))
( ( )) ( ( ))
If the errors in the observations are independent, then
and ( ) ( ( ), ( )) log[ ( ( 1) | ( ))] log ( (0))n
P x m y m P y m x mP x m Y m P x m Y m P x m Y m
P x m P y m
A X CMI x n y n P x n x n P x
With usual assumptions: the measurements are contaminated by Gaussian noise, …
, 1,2 2
00, 1 0, 1
Measurement Error (Synchron
(
i
)
z
( , ) ( ) [( ( ) ( ))] + (
ation) Model Er
( 1) ( ( )
ror (Finite Resolut
))2 2
ion)
log( ( (0))
m L m Dfm
l l a an l n a
R aR n lA X x n y n x n f x n
P x
Initial Condition
Approximating the Action A0(X)—``cost function’’
22 /2
2
( ( 1) | ( )) = ( ( 1) ( ( )))
( ( 1) ( ( ))(2 ) exp[ ]
2
D
D
P x n x n x n f x n
x n f x n
Finite Model Resolution
Gaussian Error Action
We can put the two tasks of data assimilation together by thinking about an interval from t0 through the end of measurements at tm = T and into a prediction window from T to TP.
0t
0( ( ))
( (0))
P x t
P x
ntmt T
P Pt T
( ( ) | (0), (1),..., ( ))mP x t y y y m ( ( ) | (0), (1),..., ( ))mP x t t y y y m
(0) (1) .... ( )y y y m
Observation Window Prediction Window
(0), (1),..., ( ),..., ( ) ( ),
x(m+1),.... ( ),... ( )P
x x x n x m X T
x p x T
Path :
X =
Predict ( ( ))l mh x t t
