Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture:...
Preview:
Citation preview
- Slide 1
- Lecture series: Data analysis Lectures: Each Tuesday at 16:00
(First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC,
CNR thomas.kreuz@cnr.it
http://www.fi.isc.cnr.it/users/thomas.kreuz/
- Slide 2
- Lecture 1: Example (Epilepsy & spike train synchrony), Data
acquisition, Dynamical systems Lecture 2: Linear measures,
Introduction to non-linear dynamics Lecture 3: Non-linear measures
Lecture 4: Measures of continuous synchronization Lecture 5:
Measures of discrete synchronization (spike trains) Lecture 6:
Measure comparison & Application to epileptic seizure
prediction Schedule
- Slide 3
- Example: Epileptic seizure prediction Data acquisition
Introduction to dynamical systems First lecture
- Slide 4
- Non-linear model systems Linear measures Introduction to
non-linear dynamics Non-linear measures - Introduction to phase
space reconstruction - Lyapunov exponent Second lecture
- Slide 5
- Non-linear measures - Dimension [ Excursion: Fractals ] -
Entropies - Relationships among non-linear measures Third
lecture
- Slide 6
- Characterizition of a dynamic in phase space Predictability
(Information / Entropy) Density Self-similarityLinearity /
Non-linearity Determinism / Stochasticity (Dimension) Stability
(sensitivity to initial conditions)
- Slide 7
- Dimension (classical) Number of degrees of freedom necessary to
characterize a geometric object Euclidean geometry: Integer
dimensions Object Dimension Point0 Line1 Square (Area)2 Cube
(Volume)3 N-cuben Time series analysis: Number of equations
necessary to model a physical system
- Slide 8
- Hausdorff-dimension
- Slide 9
- Box-counting
- Slide 10
- Richardson: Counter-intuitive notion that a coastline's
measured length changes with the length of the measuring stick
used. Fractal dimension of a coastline: How does the number of
measuring sticks required to measure the coastline change with the
scale of the stick?
- Slide 11
- Example: Koch-curve Some properties: - Infinite length -
Continuous everywhere - Differentiable nowhere - Fractal dimension
D=log4/log3 1.26
- Slide 12
- Strange attractors are fractals Logistic map Hnon map 2,01
- Slide 13
- Self-similarity of the logistic attractor
- Slide 14
- Generalized dimensions
- Slide 15
- Generalized entropies
- Slide 16
- Lyapunov-exponent
- Slide 17
- Summary
- Slide 18
- Motivation Measures of synchronization for continuous data
Linear measures: Cross correlation, coherence Mutual information
Phase synchronization (Hilbert transform) Non-linear
interdependences Measure comparison on model systems Measures of
directionality Granger causality Transfer entropy Todays
lecture
- Slide 19
- Motivation
- Slide 20
- Motivation: Bivariate time series analysis Three different
scenarios: Repeated measurement from one system (different times)
Stationarity, Reliability Simultaneous measurement from one system
(same time) Coupling, Correlation, Synchronization, Directionality
Simultaneous measurement from two systems (same time) Coupling,
Correlation, Synchronization, Directionality
- Slide 21
- Synchronization [Huygens: Horologium Oscillatorium. 1673]
- Slide 22
- Synchronization [Pecora & Carroll. Synchronization in
chaotic systems. Phys Rev Lett 1990]
- Slide 23
- Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] In-phase synchronization
- Slide 24
- Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] Anti-phase synchronization
- Slide 25
- Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] Synchronization with phase shift
- Slide 26
- Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] No synchronization
- Slide 27
- Synchronization [Pikovsky & Rosenblum: Synchronization.
Scholarpedia (2007)] In-phase synchronizationAnti-phase
synchronization No synchronizationSynchronization with phase
shift
- Slide 28
- Measures of synchronization SynchronizationDirectionality Cross
correlation / Coherence Mutual Information Index of phase
synchronization - based on Hilbert transform - based on Wavelet
transform Non-linear interdependence Event synchronization Delay
asymmetry Transfer entropy Granger causality
- Slide 29
- Linear correlation
- Slide 30
- Static linear correlation: Pearsons r -1 - completely
anti-correlated r = 0 - uncorrelated (linearly!) 1 - completely
correlated Two sets of data points:
- Slide 31
- Examples: Pearsons r Undefined [An example of the correlation
of x and y for various distributions of (x,y) pairs; Denis Boigelot
2011]
- Slide 32
- Cross correlation Maximum cross correlation:
- Slide 33
- Coherence Linear correlation in the frequency domain Cross
spectrum: Coherence = Normalized power in the cross spectrum Welchs
method: average over estimated periodograms of subintervals of
equal length Complex number Phase
- Slide 34
- Mutual information
- Slide 35
- Shannon entropy ~ Uncertainty Binary probabilities:In
general:
- Slide 36
- Mutual Information Marginal Shannon entropy: Joint Shannon
entropy: Mutual Information: Estimation based on k-nearest neighbor
distances: [Kraskov, Stgbauer, Grassberger: Estimating Mutual
Information. Phys Rev E 2004] Kullback-Leibler entropy compares to
probability distributions Mutual Information = KL-Entropy with
respect to independence
- Slide 37
- Mutual Information Properties: Non-negativity: Symmetry:
Minimum: Independent time series Maximum: for identical systems
Venn diagram (Set theory)
- Slide 38
- Cross correlation & Mutual Information 1.0 0.5 0.0 C max I
1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I
- Slide 39
- Phase synchronization
- Slide 40
- Definition of a phase - Rice phase - Hilbert phase - Wavelet
phase Index of phase synchronization - Index based on circular
variance - [Index based on Shannon entropy] - [Index based on
conditional entropy] Phase synchronization [Tass et al. PRL
1998]
- Slide 41
- Linear interpolation between marker events - threshold
crossings (mostly zero, sometimes after demeaning) - discrete
events (begin of a new cycle) Problem: Can be very sensitive to
noise Rice phase
- Slide 42
- Hilbert phase [Rosenblum et al., Phys. Rev. Lett. 1996]
Analytic signal: Artificial imaginary part: Instantaneous Hilbert
phase: - Cauchy principal value
- Slide 43
- Wavelet phase Basis functions with finite support Example:
complex Morlet wavelet Wavelet = Hilbert + filter [ Quian Quiroga,
Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002 ] Wavelet
phase:
- Slide 44
- Index of phase synchronization: Circular variance (CV)
- Slide 45
- Non-linear interdependence
- Slide 46
- Takens embedding theorem [F. Takens. Detecting strange
attractors in turbulence. Springer, Berlin, 1980]
- Slide 47
- Non-linear interdependences Nonlinear interdependence
SNonlinear interdependence H Synchronization Directionality
[Arnhold, Lehnertz, Grassberger, Elger. Physica D 1999]
- Slide 48
- Non-linear interdependence
- Slide 49
- Slide 50
- Slide 51
- Slide 52
- Slide 53
- Slide 54
- Slide 55
- Slide 56
- Slide 57
- Slide 58
- Event synchronization
- Slide 59
- Event synchronization Event times: Synchronicity: Event
synchronization:Delay asymmetry: [Quian Quiroga, Kreuz,
Grassberger. Phys Rev E 2002] Window: with Avoids
double-counting
- Slide 60
- Event synchronization [Quian Quiroga, Kreuz, Grassberger. Phys
Rev E 2002] Q q
- Slide 61
- Measure comparison on model systems
- Slide 62
- Measure comparison on model systems [Kreuz, Mormann, Andrzejak,
Kraskov, Lehnertz, Grassberger. Phys D 2007]
- Slide 63
- Model systems & Coupling schemes
- Slide 64
- Hnon map Introduced by Michel Hnon as a simplified model of the
Poincar section of the Lorenz model One of the most studied
examples of dynamical systems that exhibit chaotic behavior [M.
Hnon. A two-dimensional mapping with a strange attractor. Commun.
Math. Phys., 50:69, 1976]
- Slide 65
- Hnon map
- Slide 66
- Coupled Hnon maps Driver: Responder: Identical systems:
Coupling strength:
- Slide 67
- Coupled Hnon maps
- Slide 68
- Coupled Hnon systems
- Slide 69
- Rssler system designed in 1976, for purely theoretical reasons
later found to be useful in modeling equilibrium in chemical
reactions [ O. E. Rssler. An equation for continuous chaos. Phys.
Lett. A, 57:397, 1976 ]
- Slide 70
- Rssler system
- Slide 71
- Coupled Rssler systems Driver: Responder: Parameter mismatch:
Coupling strength:
- Slide 72
- Coupled Rssler systems
- Slide 73
- Slide 74
- Lorenz system Developed in 1963 as a simplified mathematical
model for atmospheric convection Arise in simplified models for
lasers, dynamos, electric circuits, and chemical reactions [ E. N.
Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130,
1963 ]
- Slide 75
- Lorenz system
- Slide 76
- Coupled Lorenz systems Driver: Responder: Small parameter
mismatch in second component Coupling strength:
- Slide 77
- Coupled Lorenz systems
- Slide 78
- Slide 79
- Noise-free case
- Slide 80
- Criterion I: Degree of monotonicity = 1 - strictly monotonic
increase M(s) = 0 - flat line (or equal decrease and increase) = -1
- strictly monotonic decrease
- Slide 81
- Degree of monotonicity: Examples Sequences: 100 values 5050
pairs Left: Monotonicity Right: # positive # negative
- Slide 82
- Comparison: No Noise
- Slide 83
- Summary: No-Noise-Comparison Results for Rssler are more
consistent than for the other systems Mutual Information slightly
better than cross correlation (Non-linearity matters) Wavelet phase
synchronization not appropiate for broadband systems (inherent
filtering looses information)
- Slide 84
- Robustness against noise
- Slide 85
- Criterion II: Robustness against noise
- Slide 86
- Example: White noise
- Slide 87
- Hnon system: White noise
- Slide 88
- Rssler system : White noise
- Slide 89
- Lorenz system: White noise
- Slide 90
- Hnon system
- Slide 91
- Comparison: White noise
- Slide 92
- Summary: White noise For systems opposite order as in the
noise-free case (Lorenz more robust then Hnon and than Rssler) the
more monotonous a system has been without noise, the less noise is
necessary to destroy this monotonicity Highest robustness is
obtained for cross correlation followed by mutual information.
- Slide 93
- Iso-spectral noise: Example
- Slide 94
- Iso-spectral noise: Fourier spectrum complex Autocorrelation
Fourier spectrum Time domain Frequency domain x (t) Amplitude
Physical phenomenon Time series
- Slide 95
- Generation of iso-spectral noise Phase-randomized surrogates:
Take Fourier transform of original signal Randomize phases Take
inverse Fourier transform Iso-spectral surrogate (By construction
identical Power spectrum, just different phases) Add to original
signal with given NSR
- Slide 96
- Lorenz system: Iso-spectral noise
- Slide 97
- Comparison: Iso-spectral noise
- Slide 98
- Summary: Iso-spectral noise Again results for Rssler are more
consistent than for the other systems Sometimes M never crosses
critical threshold (monotonicity of the noise-free case is not
destroyed by iso-spectral noise). Sometimes synchronization
increases for more noise: (spurious) synchronization between
contaminating noise-signals, only for narrow-band systems
- Slide 99
- Correlation among measures
- Slide 100
- Correlation among measures
- Slide 101
- Slide 102
- Slide 103
- Slide 104
- Summary: Correlation All correlation values rather high
(Minimum: ~0.65) Highest correlations for cross correlation and
Hilbert phase synchronization Event synchronization and Hilbert
phase synchronization appear least correlated Overall correlation
between two phase synchronization methods low (but only due to
different frequency sensitivity in the Hnon system)
- Slide 105
- Overall summary: Comparison of measures Capability to
distinguish different coupling strengths Obvious and objective
criterion exists only in some special cases (e.g., wavelet phase is
not very suitable for a system with a broadband spectrum).
Robustness against noise varies (Important criterion for noisy
data) Pragmatic solution: Choose measure which most reliably yields
valuable information (e.g., information useful for diagnostic
purposes) in test applications
- Slide 106
- Measures of directionality
- Slide 107
- Measures of directionality
- Slide 108
- Granger causality [Granger: Investigating causal relations by
econometric models and cross-spectral methods. Econometrica 37,
424-438 (1969)]
- Slide 109
- Granger causality Univariate model: Bivariate model: Model
parameters; Prediction errors; [Granger: Investigating causal
relations by econometric models and cross-spectral methods.
Econometrica 37, 424-438 (1969)] Fit via linear regression
- Slide 110
- Transfer Entropy: Conditional entropy Venn diagram (Set theory)
Conditional entropy: Mutual Information:
- Slide 111
- Transfer entropy : Conditional entropy [T. Schreiber. Measuring
information transfer. Phys. Rev. Lett., 85:461, 2000]
- Slide 112
- Transfer entropy [T. Schreiber. Measuring information transfer.
Phys. Rev. Lett., 85:461, 2000]
- Slide 113
- Transfer entropy [T. Schreiber. Measuring information transfer.
Phys. Rev. Lett., 85:461, 2000]
- Slide 114
- Motivation Measures of synchronization for continuous data
Linear measures: Cross correlation, coherence Mutual information
Phase synchronization (Hilbert transform) Non-linear
interdependences Measure comparison on model systems Measures of
directionality Granger causality Transfer entropy Todays
lecture
- Slide 115
- Measures of synchronization for discrete data (e.g. spike
trains) Victor-Purpura distance Van Rossum distance Schreiber
correlation measure ISI-distance SPIKE-distance Measure comparison
Next lecture