Mixed Effects Models for Time Series - CIMATjortega/Ombao/Ombao_MixedEffects.pdf · In multiple...

Outline

MIXED EFFECTS MODELS FOR TIME SERIES

Cristina Gorrostieta Hakmook Kang Hernando Ombao

Brown UniversityBiostatistics Section

February 16, 2011

Outline

OUTLINE OF TALK

1 SCIENTIFIC MOTIVATION

2 BACKGROUND ON MIXED EFFECTS MODELS

3 MIXED EFFECTS VAR

4 SPATIO-SPECTRAL MIXED EFFECTS MODELS

Outline

OUTLINE OF TALK

3 MIXED EFFECTS VAR

Outline

OUTLINE OF TALK

3 MIXED EFFECTS VAR

Outline

OUTLINE OF TALK

3 MIXED EFFECTS VAR

Outline of Talk Scientific Motivation Background on Mixed Effects Models Mixed Effects VAR Spatio -Spectral Mixed Effects

ANALYSIS OF BRAIN SIGNALS

Electrophysiologic data: multi-channel EEG, local fieldpotentialsHemodynamic data: fMRI time series at several ROIs

Goals of our research

Characterize dependence in a brain network

Temporal: Y1(t) ∼ [Y1(t − 1),Y2(t − 1), . . .]′

Spectral: interactions between oscillatory activities at Y1,Y2

Develop estimation and inference methods for connectivity

Investigate potential for connectivity as a biomarker

Predicting behavior

Motor intent (left vs. right movement)State of learning

Differentiating patient groups (bipolar vs. healthy children)

Connectivity between left DLPFC ⇆ right STG is greater forbipolar than healthy

Desirable Components of Model for Connectivity

Fixed EffectsCondition-specific or group-specific connectivityTest for differences across conditions or groups

Random EffectsIn multiple subjects: subject-specific random deviation fromthe group effectIn multiple trials: trial-specific random deviation from themean condition effectModel variation in connectivity betweensubjects/participants in a group; and between trials within asubject

Outline of Talk Scientific Motivation Background on Mixed Effects Models Mixed Effects VAR Spatio-Spectral Mixed Effects

MIXED EFFECTS MODELS

A Simple Regression Model

Data for one subject: {(xt ,Yt), t = 1, . . . ,T}

A simple model

Yt = f (xt) + ǫt , ǫt ∼ (0, σ2)

f (xt) = β0 + β1xt parametric

f (xt) =∑

βkψk (xt) non-parametric

β’s are fixed effects

Estimation and inference via maximum likelihood

MIXED EFFECTS MODELS

Multi-subject data: {(xnt ,Y

nt ); t = 1, . . . ,T ;n = 1, . . . ,N}

Fixed effects only model

Y nt = β0 + β1xn

t + ǫnt

Y nt =

βkψ(xnt ) + ǫnt

Mixed effects model

Y nt = βn

0 + βn1 xn

t + ǫnt

Y nt =

βnkψ(x

nt ) + ǫnt

Decompose βnk = βk + bn

k ; where bnk ∼ (0, τ2)

Eβnk = βk – fixed effect (average across all subjects)

bnk – subject-specific random deviation from the group effect

ILLUSTRATION: ESTIMATING THE HEMODYNAMIC

RESPONSE IN FMRI

Goal is to estimate the brain hemodynamic responsefunction in olfaction stimuli

Standard HRF are estimated from visual stimuli

In visual stimuli rise and decay are fast

In olfaction, rise is slow, peak is persistent and decay isslow

Experiment: 30 fMRI trials; each trial 20 seconds; TR = 1second

RESPONSE IN FMRI

fMRI time series for several trials

RESPONSE IN FMRI

{φk (t)} B-splines basis

RESPONSE IN FMRI

Time series for trial n

Y n(t) = f n(t) + ǫn(t)

HRF for trial n

f n(t) = f (t) + δn(t)

f (t) =∑

βkφk (t)

δn(t) =∑

bnkφk (t)

f (t) - over-all olfaction HRFδn trial-specific deviation from the over-all HRFImplemented in R (nlmix) and SAS (proc mixed)

RESPONSE IN FMRI

Trial-specific HRF estimate

RESPONSE IN FMRI

Over-all HRF estimate

ESTIMATION

Model: Yn = Xnβ + Znbn + ηn

Between-subjects Random Effect: bn ∼ N(0,D)

Within-subject error: ηn ∼ N(0,V )

Unconditional distribution

Yn ∼ N(Xnβ,ZnDZ′

n + V )

Conditional distribution (subject n)

Yn | bn ∼ N(Xnβ + Znbn,V )

The subject-specific mean function

E [Yn | bn] = Xnβ + Znbn

ESTIMATION

Marginal Model Yn = Xnβ + ǫn

ǫn = Znbn + ηn

ǫn ∼ N(0,Σ(θ))

Yn ∼ N(Xnβ,Σ(θ))

Estimates β̂ and θ̂ obtained by REML

Random effects estimated by empirical Bayes

Prior bn ∼ N(0,D)Likelihood Yn ∼ N(Xnβ,ZnDZ

n + σ2I)Posterior ∝ Prior × LikelihoodPlug in estimates D̂, β̂, σ̂2

b̂n =[Z′

nZn + D̂−1σ̂2]−1

[Yn − Xnβ̂

Outline of Talk Scientific Motivation Background on Mixed Ef fects Models Mixed Effects VAR Spatio-Spectral Mixed Effects

MIXED-EFFECTS MODELS IN TIME SERIES

Experiment

N = 15 right-handed college students

Experiment: subjects see visual targets and must movejoystick

Two Conditions

Free choice - subject freely chooses any targetInstructed - subject must choose the specified target

Regions of interest (7 areas that show highest differentialactivation)

PFC, SMA, etc.

0.22%0

Total of R ROIs

Y nr (t) the fMRI time series at the ROI r for subject n

Entire network: Yn(t) = [Y n1 (t), . . . ,Y

nR(t)]

General additive model

Yn(t) = Fn(t) + En(t)

The componentsFn(t) - mean (deterministic) componentEn(t) - stochastic component

The mean component F n(t)

Decomposition

Fn(t) = Dn(t) + Mn(t) + βn1 ⊗ X1(t) + . . .+ βn

C ⊗ XC(t),

The mean component includes systematic changes in theBOLD signal that is due to

Scanner driftPhysiological signals of non-interest (e.g., cardiac andrespiratory)Experimental conditions

Methods for estimating the mean component

Worsley and Friston, 1995;Nichols and Holmes, 2002.

Implemented inStatistical Parametric Mapping (SPM)FMRIB Statistical Laboratory (FSL)Analysis of Functional NeuroImages (AFNI)

The stochastic component E n(t)

En(t) captures between-ROI connectivity

Cov[Yn(t + h),Yn(t)] = Cov[Fn(t + h) + En(t + h),Fn(t) + En(t)]

= Cov[En(t + h),En(t)]

En(t) cannot be observed directly; we use the residuals:

En(t) = Yn(t)− Fn(t)

Rn(t) = Yn(t)− F̂n(t)

Vector Auto-Regressive Model

E(t) = [E1(t),E2(t)]′

VAR(1) Model

E(t) = Φ1E(t − 1) + η(t)

Φ1 lag-1 connectivity matrix; components φkℓ

VAR(1) Equations

E1(t) = φ11E1(t − 1) + φ12E2(t − 1) + η1(t)

E2(t) = φ21E1(t − 1) + φ22E2(t − 1) + η2(t)

Conditional MLE

Model E(t) = Φ1E(t − 1) + η(t), t = 1, . . . ,TDefine E(t − 1) = {E(t − 1),E(t − 2), . . .}Suppose η(t) iid N(0,Σ)Condition on past data E(t − 1)

E(t) | E(t − 1) ∼ N(Φ1X(t − 1),Σ)

Conditional likelihood function

LC [Φ1|E(1)] = f (E(2)|E(1))× . . . × f (E(T ) | E(T − 1)))

Conditional approach leads to a closed form for theestimator of Φ1

Full likelihood - no closed form

The ME-VAR(1) Model

E(n)(t) =[Φ1,kW1(t) + Φ2,kW2(t) + b(n)

]E(n)(t − 1) + e(n)(t)

W�(t) is the indicator functionWhen condition 1 is active then W1(t) = 1 and W2(t) = 0When condition 2 is active then W1(t) = 0 and W2(t) = 1

b(n)1 models between-subject variation in connectivity

b(n)1 ∼ (0, σ2

Subject-specific connectivity matrix (condition on b(n)1 )

When W1(t) = 1, the connectivity matrix for subject n is

Φ1,1 + b(n)1

When W2(t) = 1, the connectivity matrix for subject n is

Φ1,2 + b(n)1

The model can be utilized to test for

Lagged dependence between each pair of ROIs

H0 : Φ1,1 = 0,Φ1,2 = 0

Granger causality in each experimental conditionTesting for differences between conditions

H0 : ∆1 = Φ1,1 −Φ1,2 = 0

CONNECTIVITY ANALYSIS OF FMRI DATA

Step 0. Selection of pre-defined ROIs.

Step 1. Estimate the mean function at each ROI Fn(t).

Step 2. Fit the ME-VAR model to the residuals Rn(t).

Fitting the ME-VAR Model

Optimal lag over was p∗ = 2 using the BayesianInformation Criterion (Pmax = 8)

In physical time: TR = 2 seconds; optimal delay/lag is 4seconds.

Computations were carried out in SAS proc mixed (we useexisting machinery!)

PMd PMd

CONCLUSION

Statistical vs Clinical significance

Connectivity parameter of 0.018 can be statisticallysignificant. Is it clinically meaningful?

How do we measure clinical significance?

Impact of increase (or decrease) in connectivity measureon some clinical outcome

Should we be testing for H0 : θ = 0 or H0 : |θ| > η?

CONCLUSION

Granger causality – not physiological causality

Connectivity in the hemodynamic activity – not necessarilyneuronal — temporal ordering could be switched becauseof the poor temporal resolution in fMRI

SPATIO-SPECTRAL MIXED EFFECTS MODELS

Spatio-temporal models

In fMRI, spatial correlation adds computationalcomplications!

Current approaches

Bowman (2005, 2007, 2008); Worsley (1999); Valdes-Sosaet al. (2004)Spatial - parametric; “functional" distanceBayesian hierarchial4D-Wavelet packetsPartial least squares

Spatio-Spectral Model

With H. Kang (PhD student) [John Van Ryzin Award 2011]

Goals:

Estimate ROI-specific activationUse all local and global information

Approach

Spectral domain: Fourier coefficients are approxuncorrelatedSpatio-spectral covariance matrix is block diagonal -simplified!Multi-scale correlation - local (within ROI) and global(between ROIs)

ROI3ROI2

Spatio-temporal Model

Ycv (t) =P∑

[βpc + bp

cv ]Xp(t) + dc(t) + ǫcv (t)

βpc is the ROI-specific fixed effect due to stimulus p;

bpcv is a zero-mean voxel-specific random deviation

local spatial covariance between voxels within an ROICov(bp

cv , bpc′v ′) = δ(c − c′)ψb(‖ v − v ′ ‖)

dc(t) is a zero-mean ROI-specific signalglobal spatial covariationCov(dc(t), dc′ (t)) = ψd (c, c′)

ǫcv (t) is the within-voxel temporal noise

Problem: Spatio-temporal covariance structure iscomplicated!

A solution: Fourier transform

X(t) stationary time series with covariance matrix Σd(ωk ) =

∑t X(t)exp(−i2πωk t) Fourier coefficient

I(ωk ) = |d(ωk )|2

I(ω1), . . . , I(ωM) are approx uncorrelated

Ycv (t) time series at voxel v in ROI c

Ycv (ω) corresponding Fourier coefficient

Spatio-Spectral Model

Ycv (ωk ) =P∑

[βpc + bp

cv ]Xp(ωk ) + dc(ωk ) + ǫcv (ωk )

βpc is the ROI-specific fixed effect due to stimulus p

bpcv is a zero-mean voxel-specific random effect

dc(t) is a zero-mean ROI-specific random effect

Mixed Effects Models for Time Series - CIMATjortega/Ombao/Ombao_MixedEffects.pdf · In multiple...

Documents

Mean Absolute Deviation - Loudoun County Public Schools€¦ · and interpret mean absolute deviation, ... Sample Standard Deviation Population Standard Deviation ... 12, 12, 14,

Standard deviation (3)

DEVIATION SETTLEMENT.pdf

DEVIATION #1

Introduction to Spectral Analysis - CIMATjortega/Ombao/Ombao_CIMAT_1.pdf · 2011-03-18 · Some References for Time Series Analysis Brillinger (1981) - Theory for Spectral Analysis

VITRIFIE… · 10. 18. Technical Characteristics Deviation in Length and Width Deviation in Thickness Deviation Straiotness of sides Deviation Rectangu"arity

· 10. 18. Technical Characteristics Deviation in Length and Width Deviation in Thickness Deviation Straiotness of sides Deviation Rectangu"arity

Deviation & Foregrounding

1 News about the future and ⁄uctuations - dspace.mit.edu file0 5 10 15 20 25 30 35 40 1.4 1.2 1 0.8 0.6 0.4 0.2 0 % Deviation % Deviation % Deviation % Deviation % Deviation quarters

LT8920 - Deviation

Table of Contents 1. Standard Deviation. 1. Standard Deviation Essential Question – What is standard deviation and what does it measure?

Deviation Settlement

CFO FORUM MEETING MOSSEL BAY...Standard Deviation and Variance Deviation just means how far from the normal Standard Deviation The Standard Deviation is a measure of how spread out

- Standard deviation : - Characteristics of standard deviation

Mean square deviation Root mean square deviation Variance Standard deviation

Extensive analog and digital modulations Amplitude range ... · FREQUENCY MODULATION Deviation: 1mHz – 1MHz subject to carrier frequency Deviation Setting Resolution:1mHz Deviation

User’s Manual · 2019-10-03 · - Allan deviation (ADEV) typically less than 5 E-14 at t=1s - Modified Allan deviation (MDEV), Hadamard deviation (HDEV), and time deviation (TDEV)

Eso deviation

University of California at Irvine Department of Statisticshombao/Ombao_UPLB_IBS_Dec_2012.pdf · SPATIO-TEMPORAL MODELS WITH APPLICATIONS TO HEALTH DATA Hernando Ombao University

Overall Deviation