ERPWAVELAB 1st International Summer School in Biomedical Engineering1st International Summer School...

Morten MørupTechnical University of Denmark

mm@imm.dtu.dk

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006

Analyzing the Wavelet Transformed EEG using Non-negative Matrix and Tensor Factorization

An introduction to ERPWAVELAB

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006

mm@imm.dtu.dk

Parts of the work done in collaboration with

Lars Kai Hansen, ProfessorDepartment of Signal Processing

Informatics and Mathematical Modeling,Technical University of Denmark

Sidse M. Arnfred, Dr. Med. PhDCognitive Research Unit

Hvidovre HospitalUniversity Hospital of Copenhagen

mm@imm.dtu.dk

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006

•The continuous wavelet transform and measures of the event related ERP in the time-frequency domain•Introduction to NMF and extensions to tensor decompositions (PARAFAC & TUCKER)•Accessing significance•A demonstration of ERPWAVELAB•Discussion

OUTLINE

mm@imm.dtu.dk

frequency

Continuous Wavelet transform

Complex Morlet wavelet - Real part - Complex part

Absolute value of wavelet coefficient

Captures frequency changes through time

mm@imm.dtu.dk

Continuous Wavelet transform (continued)

time-frequency

epochtimechannel X epochfrequencytimechannel X

mm@imm.dtu.dk

The Vector strength

Vectors coherent, i.e. correlated Vectors incoherent, i.e. uncorrelated

Vector strength a measure of coherence

mm@imm.dtu.dk

Measures of the event related ERP in the time-frequency domain

mm@imm.dtu.dk

Measures of the event related ERP in the time-frequency domain (cont.)

Since scalp works as low pass filter it is customary to normalize X before calculating each measure. Frequently used normalizations are:(where tb are points in the baseline region and Tb the total number of baseline samples)

fntfcX

ntfcXT

ntfcXb

),,,(or

),,,(1

mm@imm.dtu.dk

ERPWAVELAB demonstration, tutorial dataset 1

WTav avWT INDUCED

mm@imm.dtu.dk

f’=f, t’=t

mm@imm.dtu.dk

Multi-channel decomposition

time-frequency activities appear to be similar across channels but varying in strength.

Motivates to decompose the activity into similar time-frequency signatures varying in strength in the recording channels.Thus, this form of decomposition is primarily useful for data exploratory purposes giving very easy summaries of what types of activities are present in the data. Only when the measures of interest can be assumed linear and no cancellation between sources are present the decomposition can also reveal the underlying true sources.

mm@imm.dtu.dk

Factor Analysis

Spearman ~1900

Vtests x subjects Wtests x intelligencesHintelligencesxsubject

Subjects Subjects

Non-negative Matrix Factorization (NMF): VWH s.t. Wi,d,Hd,j0

(~1970 Lawson, ~1995 Paatero, ~2000 Lee & Seung)

mm@imm.dtu.dk

Non-negative matrix factorization (NMF)NMF: VWH s.t. Wi,d,Hd,j0

Multiplicative updates: Let C be a given cost function

Positive termNegative term

mm@imm.dtu.dk

Non-negative matrix factorization (NMF)

(Lee & Seung - 2001)

NMF gives Part based representation (Lee & Seung – Nature 1999)

mm@imm.dtu.dk

The NMF decomposition is not unique

Simplical Cone

NMF only unique when data adequately spans the positive orthant (Donoho & Stodden - 2004)

HWH)(WP)(PWHV -1 ~~

Convex Hull

Positive Orthant

Simplical Cones

mm@imm.dtu.dk

Sparse Coding NMF (SNMF)

(Eggert & Körner, 2004) (Mørup & Schmidt, 2006)

mm@imm.dtu.dk

Why sparseness?

• Ensures uniqueness• Eases interpretability

(sparse representation factor effects pertain to fewer dimensions)

• Can work as model selection(Sparseness can turn off excess factors by letting them become zero)

• Resolves over complete representations (when model has many more free variables than data points)

mm@imm.dtu.dk

time-frequency Subjec

ts/Con

dition

Often extra modalities such as subjects, conditions and trials are present, consequently the data forms a tensor.

Need for tensor decomposition

mm@imm.dtu.dk

Higher Order Non-negative Matrix Factorization

Factor Analysis PARAFAC TUCKER

ddidiii

321321 di

ddidiiii AAA

2211321321

jjijijjjiii AAACV

1211111321 ji

jjjjijijjjiii AAA

mm@imm.dtu.dk

Uniqueness• Although PARAFAC in general is unique under mild conditions, the

proof of uniqueness by Kruskal is based on k-rank*. However, the k-rank does not apply for non-negativity**.

• TUCKER model is not unique, thus no guaranty of uniqueness.Imposing sparseness useful in order to achieve unique decompositions

Tensor decompositions known to have problems with degeneracy, however when imposing non-negativity degenerate solutions can’t occur***

*) k-rank: The maximum number of columns chosen by random of a matrix certain to be linearly independent. **) L.-H. Lim and G.H. Golub, 2006.***) See L.-H. Lim - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf

mm@imm.dtu.dk

Example why Non-negative PARAFAC isn’t unique

)3()2()1(

ranknegativeNon

satisfiedFKKKconditionKruskal

mm@imm.dtu.dk

PARAFAC model estimation

321321 di

ddidiiii AAA

Thus, the PARAFAC model is by the matricizing operationThus, the PARAFAC model is by the matricizing operationestimated straight forward from regular NMF estimationestimated straight forward from regular NMF estimation

JJ BABABABA 2211

mm@imm.dtu.dk

TUCKER

2211321321

jjijijjjiii AAACV

TUCKER model estimation

T23)1(

)( 123 AAA CV vecvec

mm@imm.dtu.dk

Algorithm outline (TUCKER) (PARAFAC follows by setting C=I)

mm@imm.dtu.dk

Accessing Significance

• Comparison to known distribution • Bootstrapping• Cross validation

mm@imm.dtu.dk

Comparison to known distribution

(Mardia, Directional Statistics)

Rayleigh distributedRed: Theoretical mean value of N-½

Black: Mean value estimated by bootstrapping

Normal distributed

Random ITPC and ERPCOH corresponds to a random walk in the complex planethus is Rayleigh distributed.

mm@imm.dtu.dk

Bootstrapping

1) Randomly select Data from the epochs to form new datasets (each epoch might be represented 0, 1 or several times in the datasets). 2) Calculate the measure of interest for each of these datasets.3) Evaluate the values found to the distribution of values found by the bootstrap datasets.

mm@imm.dtu.dk

Cross validation

• Split dataset into exploratory and confirmatory datasets.

• Find significant activity in exploratory dataset

• See if this activity is also significant in confirmatory dataset

Correcting for multiple comparison by bootstrapping very expensive and often too conservative. Thus Cross validation useful.

mm@imm.dtu.dk

– Dataset generation– Single subject analysis

Artifact rejection in the time frequency domainNMF decompositionCross coherence tracking

– Multi subject analysisClusteringAnalysis of Variance (ANOVA)Tensor decomposition

ERPWAVELAB Tutorial:

The toolbox is free to download from www.erpwavelab.com

mm@imm.dtu.dk

Epilog: Some History of PARAFAC and EEG

• Harshman (1970) (Suggested its use on EEG)• Möcks (1988) (Topographic Component Analysis)

ERP of (channel x time x subject)• Field and Graupe (1991)

ERP of (channel x time x subject)• Miwakeichi et al. (2004)

EEG of (channel x time x frequency)• Mørup et al. (2005)

ERP of ITPC (channel x time x frequency x subject x condition)

mm@imm.dtu.dk

References

Bro, R., 1998. Multi-way Analysis in the Food Industry: Models, algorithms and Applications. Amsterdam, Copenhagen.Bro, R.,Jong, S. D., 1997. A fast non-negativity-constrained least squares algorithm. J. Chemom. 11, 393–401Carroll, J. D. and Chang, J. J. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition, Psychometrika 35 1970 283—319Delorme, A.,Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134, 9-21Donoho, D. and Stodden, V. When does non-negative matrix factorization give a correct decomposition into parts? NIPS2003Eggert, J. and Korner, E. Sparse coding and NMF. In Neural Networks volume 4, pages 2529-2533, 2004Field, Aaron S.; Graupe, Daniel “Topographich Component (Parallel Factor) analysis of Multichannel Evoked Potentials: Practical Issues in Trilinear Spatiotemporal Decomposition” Brain Topographa, Vol. 3, Nr. 4, 1991 Fiitzgerald, D. et al. Non-negative tensor factorization for sound source separation. In proceedings of Irish Signals and Systems Conference, 2005Kruskal, J.B. Three-way analysis: rank and uniqueness of trilinear decompostions, with application to arithmetic complexity and statistics. Linear Algebra Appl., 18: 95-138, 1977Harshman, R. A. Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis},UCLA Working Papers in Phonetics 16 1970 1—84Herrmann, C. S., Grigutsch, M.,Busch, N. A., 2005. EEG oscillations and wavelet analysis.Lathauwer, Lieven De and Moor, Bart De and Vandewalle, Joos MULTILINEAR SINGULAR VALUE DECOMPOSITION.SIAM J. MATRIX ANAL. APPL.2000 (21)1253–1278Lee, D.D. and Seung, H.S. Algorithms for non-negative matrix factorization. In NIPS, pages 556-462, 2000Lee, D.D and Seung, H.S. Learning the parts of objects by non-negative matrix factorization, NATURE 1999Lim, Lek-Heng - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf

Lim, L.-H. and Golub, G.H., "Nonnegative decomposition and approximation of nonnegative matrices and tensors," SCCM Technical Report, 06-01, forthcoming, 2006.Mardia, K. V.,Jupp, P. E., 1999. Directional Statistics. WILEY & SONS 76-77Miwakeichi, F., Martinez-Montes, E., Valdes-Sosa, P. A., Nishiyama, N., Mizuhara, H., Yamaguchi, Y., 2004. Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis. Neuroimage 22, 1035-1045.

Möcks, J., 1988. Decomposing event-related potentials: a new topographic components model. Biol. Psychol. 26, 199-215.

Mørup, M and Hansen, L.K and Herman, C.S. and Parnas, Josef and Arnfrede, Sidse M. “Parallel Factor Analysis as an exploratory tool for wavelet transformed event –related EEG” NeuroImage 20, 938-947 (2006)

Mørup, M. and Hansen, L.K.and Arnfred, S.M.Decomposing the time-frequency representation of EEG using nonnegative matrix and multi-way factorization Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006a

Mørup, M. and Schmidt, M.N. Sparse non-negative matrix factor 2-D deconvolution. Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006b

Mørup, M. and Hansen, L.K.and Arnfred, S.M. Algorithms for Sparse Higher Order Non-negative Matrix Factorization (HONMF), Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006e

Tamara G. Kolda Multilinear operators for higher-order decompositions technical report Sandia national laboratory 2006 SAND2006-2081.

Tucker, L. R. Some mathematical notes on three-mode factor analysis Psychometrika 31 1966 279—311

Welling, M. and Weber, M. Positive tensor factorization. Pattern Recogn. Lett. 2001

ERPWAVELAB 1st International Summer School in Biomedical Engineering1st International Summer School...

Documents

DEVVARTS 1st International Workshop on

2 nd International Conference on Biomedical Ontology (ICBO’11)

1st INTERNATIONAL VERMOUTH FAIR

1st International Emergency Response Conference

I educate international professionals in biomedical technologies

Biomedical Informatics - International Joint Conference on

International Journal of Biomedical Engineering (vol 2 issue-2)

International Journal of Biomedical Engineering vol 2 issue-1

International Forum of Biomedical Materials - … Circular.pdf · International Forum of Biomedical Materials — Biomaterials Interfaces and Nanobiomaterials Aug 1 – 3, 2016, Hangzhou,

1st International Conference on Electrolysis

CIOMS International Ethical Guidelines for Biomedical Resear

1ST International Choral Conducting Competition ROMANO ... · 1ST International Choral Conducting Competition ROMANO GANDOLFI Competition Rules I. GENERAL TERMS 1. The 1st International

International Forum of Biomedical Materials · International Forum of Biomedical Materials — Biomaterials Interfaces and Nanobiomaterials Aug 1 – 3, 2016, Hangzhou, China During

Spanish CanSat Activities 1st International CanSat ... - cnes · Spanish CanSat Activities 1st International CanSat Competition LEEM-UPM. European CanSat Workshop 1st October 2009

1st International Conference International Conference on ... · 1st International Conference International Conference on Pavement Preservation by R. Gary Hicks CP2 Center-CSU, Chico

Fogarty International Center The Evolving Biomedical

1 Biomedical Optical Imaging - Purdue Engineeringbouman/publications/tutorials/ISBI... · 2007 IEEE International Symposium on Biomedical Imaging April ... Introduction to Biomedical

Advanced biomedical devices (international business)

International Workshop on Biomedical Engineering at …€¦ · International Workshop on Biomedical Engineering ... Biomedical Engineering at Osaka Institute of ... high refractive

2021 International Symposium on Biomedical Engineering and