Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture:...
115
Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC, CNR [email protected]http://www.fi.isc.cnr.it/users/thomas.kreuz /
Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC, CNR [email protected]
Lecture series: Data analysis Lectures: Each Tuesday at 16:00
(First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC,
CNR [email protected]
http://www.fi.isc.cnr.it/users/thomas.kreuz/
Slide 2
Lecture 1: Example (Epilepsy & spike train synchrony), Data
acquisition, Dynamical systems Lecture 2: Linear measures,
Introduction to non-linear dynamics Lecture 3: Non-linear measures
Lecture 4: Measures of continuous synchronization Lecture 5:
Measures of discrete synchronization (spike trains) Lecture 6:
Measure comparison & Application to epileptic seizure
prediction Schedule
Slide 3
Example: Epileptic seizure prediction Data acquisition
Introduction to dynamical systems First lecture
Slide 4
Non-linear model systems Linear measures Introduction to
non-linear dynamics Non-linear measures - Introduction to phase
space reconstruction - Lyapunov exponent Second lecture
Slide 5
Non-linear measures - Dimension [ Excursion: Fractals ] -
Entropies - Relationships among non-linear measures Third
lecture
Slide 6
Characterizition of a dynamic in phase space Predictability
(Information / Entropy) Density Self-similarityLinearity /
Non-linearity Determinism / Stochasticity (Dimension) Stability
(sensitivity to initial conditions)
Slide 7
Dimension (classical) Number of degrees of freedom necessary to
characterize a geometric object Euclidean geometry: Integer
dimensions Object Dimension Point0 Line1 Square (Area)2 Cube
(Volume)3 N-cuben Time series analysis: Number of equations
necessary to model a physical system
Slide 8
Hausdorff-dimension
Slide 9
Box-counting
Slide 10
Richardson: Counter-intuitive notion that a coastline's
measured length changes with the length of the measuring stick
used. Fractal dimension of a coastline: How does the number of
measuring sticks required to measure the coastline change with the
scale of the stick?
Strange attractors are fractals Logistic map Hnon map 2,01
Slide 13
Self-similarity of the logistic attractor
Slide 14
Generalized dimensions
Slide 15
Generalized entropies
Slide 16
Lyapunov-exponent
Slide 17
Summary
Slide 18
Motivation Measures of synchronization for continuous data
Linear measures: Cross correlation, coherence Mutual information
Phase synchronization (Hilbert transform) Non-linear
interdependences Measure comparison on model systems Measures of
directionality Granger causality Transfer entropy Todays
lecture
Slide 19
Motivation
Slide 20
Motivation: Bivariate time series analysis Three different
scenarios: Repeated measurement from one system (different times)
Stationarity, Reliability Simultaneous measurement from one system
(same time) Coupling, Correlation, Synchronization, Directionality
Simultaneous measurement from two systems (same time) Coupling,
Correlation, Synchronization, Directionality
Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] No synchronization
Slide 27
Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] In-phase synchronizationAnti-phase
synchronization No synchronizationSynchronization with phase
shift
Slide 28
Measures of synchronization SynchronizationDirectionality Cross
correlation / Coherence Mutual Information Index of phase
synchronization - based on Hilbert transform - based on Wavelet
transform Non-linear interdependence Event synchronization Delay
asymmetry Transfer entropy Granger causality
Slide 29
Linear correlation
Slide 30
Static linear correlation: Pearsons r -1 - completely
anti-correlated r = 0 - uncorrelated (linearly!) 1 - completely
correlated Two sets of data points:
Slide 31
Examples: Pearsons r Undefined [An example of the correlation
of x and y for various distributions of (x,y) pairs; Denis Boigelot
2011]
Slide 32
Cross correlation Maximum cross correlation:
Slide 33
Coherence Linear correlation in the frequency domain Cross
spectrum: Coherence = Normalized power in the cross spectrum Welchs
method: average over estimated periodograms of subintervals of
equal length Complex number Phase
Mutual Information Marginal Shannon entropy: Joint Shannon
entropy: Mutual Information: Estimation based on k-nearest neighbor
distances: [Kraskov, Stgbauer, Grassberger: Estimating Mutual
Information. Phys Rev E 2004] Kullback-Leibler entropy compares to
probability distributions Mutual Information = KL-Entropy with
respect to independence
Slide 37
Mutual Information Properties: Non-negativity: Symmetry:
Minimum: Independent time series Maximum: for identical systems
Venn diagram (Set theory)
Slide 38
Cross correlation & Mutual Information 1.0 0.5 0.0 C max I
1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I
Slide 39
Phase synchronization
Slide 40
Definition of a phase - Rice phase - Hilbert phase - Wavelet
phase Index of phase synchronization - Index based on circular
variance - [Index based on Shannon entropy] - [Index based on
conditional entropy] Phase synchronization [Tass et al. PRL
1998]
Slide 41
Linear interpolation between marker events - threshold
crossings (mostly zero, sometimes after demeaning) - discrete
events (begin of a new cycle) Problem: Can be very sensitive to
noise Rice phase
Slide 42
Hilbert phase [Rosenblum et al., Phys. Rev. Lett. 1996]
Analytic signal: Artificial imaginary part: Instantaneous Hilbert
phase: - Cauchy principal value
Slide 43
Wavelet phase Basis functions with finite support Example:
complex Morlet wavelet Wavelet = Hilbert + filter [ Quian Quiroga,
Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002 ] Wavelet
phase:
Slide 44
Index of phase synchronization: Circular variance (CV)
Measure comparison on model systems [Kreuz, Mormann, Andrzejak,
Kraskov, Lehnertz, Grassberger. Phys D 2007]
Slide 63
Model systems & Coupling schemes
Slide 64
Hnon map Introduced by Michel Hnon as a simplified model of the
Poincar section of the Lorenz model One of the most studied
examples of dynamical systems that exhibit chaotic behavior [M.
Hnon. A two-dimensional mapping with a strange attractor. Commun.
Math. Phys., 50:69, 1976]
Rssler system designed in 1976, for purely theoretical reasons
later found to be useful in modeling equilibrium in chemical
reactions [ O. E. Rssler. An equation for continuous chaos. Phys.
Lett. A, 57:397, 1976 ]
Slide 70
Rssler system
Slide 71
Coupled Rssler systems Driver: Responder: Parameter mismatch:
Coupling strength:
Slide 72
Coupled Rssler systems
Slide 73
Slide 74
Lorenz system Developed in 1963 as a simplified mathematical
model for atmospheric convection Arise in simplified models for
lasers, dynamos, electric circuits, and chemical reactions [ E. N.
Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130,
1963 ]
Slide 75
Lorenz system
Slide 76
Coupled Lorenz systems Driver: Responder: Small parameter
mismatch in second component Coupling strength:
Slide 77
Coupled Lorenz systems
Slide 78
Slide 79
Noise-free case
Slide 80
Criterion I: Degree of monotonicity = 1 - strictly monotonic
increase M(s) = 0 - flat line (or equal decrease and increase) = -1
- strictly monotonic decrease
Summary: No-Noise-Comparison Results for Rssler are more
consistent than for the other systems Mutual Information slightly
better than cross correlation (Non-linearity matters) Wavelet phase
synchronization not appropiate for broadband systems (inherent
filtering looses information)
Slide 84
Robustness against noise
Slide 85
Criterion II: Robustness against noise
Slide 86
Example: White noise
Slide 87
Hnon system: White noise
Slide 88
Rssler system : White noise
Slide 89
Lorenz system: White noise
Slide 90
Hnon system
Slide 91
Comparison: White noise
Slide 92
Summary: White noise For systems opposite order as in the
noise-free case (Lorenz more robust then Hnon and than Rssler) the
more monotonous a system has been without noise, the less noise is
necessary to destroy this monotonicity Highest robustness is
obtained for cross correlation followed by mutual information.
Slide 93
Iso-spectral noise: Example
Slide 94
Iso-spectral noise: Fourier spectrum complex Autocorrelation
Fourier spectrum Time domain Frequency domain x (t) Amplitude
Physical phenomenon Time series
Slide 95
Generation of iso-spectral noise Phase-randomized surrogates:
Take Fourier transform of original signal Randomize phases Take
inverse Fourier transform Iso-spectral surrogate (By construction
identical Power spectrum, just different phases) Add to original
signal with given NSR
Slide 96
Lorenz system: Iso-spectral noise
Slide 97
Comparison: Iso-spectral noise
Slide 98
Summary: Iso-spectral noise Again results for Rssler are more
consistent than for the other systems Sometimes M never crosses
critical threshold (monotonicity of the noise-free case is not
destroyed by iso-spectral noise). Sometimes synchronization
increases for more noise: (spurious) synchronization between
contaminating noise-signals, only for narrow-band systems
Slide 99
Correlation among measures
Slide 100
Correlation among measures
Slide 101
Slide 102
Slide 103
Slide 104
Summary: Correlation All correlation values rather high
(Minimum: ~0.65) Highest correlations for cross correlation and
Hilbert phase synchronization Event synchronization and Hilbert
phase synchronization appear least correlated Overall correlation
between two phase synchronization methods low (but only due to
different frequency sensitivity in the Hnon system)
Slide 105
Overall summary: Comparison of measures Capability to
distinguish different coupling strengths Obvious and objective
criterion exists only in some special cases (e.g., wavelet phase is
not very suitable for a system with a broadband spectrum).
Robustness against noise varies (Important criterion for noisy
data) Pragmatic solution: Choose measure which most reliably yields
valuable information (e.g., information useful for diagnostic
purposes) in test applications
Slide 106
Measures of directionality
Slide 107
Measures of directionality
Slide 108
Granger causality [Granger: Investigating causal relations by
econometric models and cross-spectral methods. Econometrica 37,
424-438 (1969)]
Slide 109
Granger causality Univariate model: Bivariate model: Model
parameters; Prediction errors; [Granger: Investigating causal
relations by econometric models and cross-spectral methods.
Econometrica 37, 424-438 (1969)] Fit via linear regression
Transfer entropy : Conditional entropy [T. Schreiber. Measuring
information transfer. Phys. Rev. Lett., 85:461, 2000]
Slide 112
Transfer entropy [T. Schreiber. Measuring information transfer.
Phys. Rev. Lett., 85:461, 2000]
Slide 113
Transfer entropy [T. Schreiber. Measuring information transfer.
Phys. Rev. Lett., 85:461, 2000]
Slide 114
Motivation Measures of synchronization for continuous data
Linear measures: Cross correlation, coherence Mutual information
Phase synchronization (Hilbert transform) Non-linear
interdependences Measure comparison on model systems Measures of
directionality Granger causality Transfer entropy Todays
lecture
Slide 115
Measures of synchronization for discrete data (e.g. spike
trains) Victor-Purpura distance Van Rossum distance Schreiber
correlation measure ISI-distance SPIKE-distance Measure comparison
Next lecture