DISCOS spm mcp - Coma Science Group webpage. Phillips, Centre de Recherches du Cyclotron, ULg, Belgium Multiple Multiple comparison problemcomparison problem DISCOS SPM course, CRC,

C. Phillips, Centre de C. Phillips, Centre de RecherchesRecherches du Cyclotron, ULg, Belgiumdu Cyclotron, ULg, Belgium

Multiple comparison problemMultiple Multiple comparisoncomparison problemproblem

DISCOS SPM course, CRC, Liège, 2009DISCOS SPM course, CRC, LiDISCOS SPM course, CRC, Lièège, 2009ge, 2009

Based on slides from: T. Nichols

ContentsContentsContents

• Recap & Introduction

• Inference & multiple comparison

• « Take home » message

•• RecapRecap & Introduction & Introduction

•• InferenceInference & multiple & multiple comparisoncomparison

•• «« TakeTake homehome »» messagemessage

realignment &motion

correction

realignment &motion

correction

smoothingsmoothingnormalisationnormalisation

General Linear Modelmodel fittingstatistic image

General Linear Modelmodel fittingstatistic image

corrected p-valuesparameter estimates

anatomical reference

kernel

image data

designmatrix

StatisticalParametric Map

correction for multiple comparisons

correction for multiple comparisons

Random effect analysisRandom effect analysis

Dynamic causal modelling,

Functional & effective connectivity, PPI, ...

Dynamic causal modelling,

Functional & effective connectivity, PPI, ...

Voxel by voxel statistics…Voxel by voxel statisticsVoxel by voxel statistics……

parameter estimation

hypothesis test

statistic image or SPM

statisticTim

e

Intensity

Tim

e

single voxeltime series

single voxeltime series

model specification

General Linear Model (in SPM)General Linear Model General Linear Model (in SPM)(in SPM)

Auditory words every 20s

SPMFSPMF

0 time 0 time secssecs 30 30

Sampled every TR = 1.7sDesign matrix,Design matrix, XX

[[ƒƒ11(u)(u)⊗⊗x(t) |x(t) |ƒƒ22(u)(u)⊗⊗x(t) |...]x(t) |...]

(Orthogonalised) (Orthogonalised) Gamma functionsGamma functions ƒƒii(u) (u) of of peristimulusperistimulus timetime uu




• Single/multiple voxel inference

• Family wise error rate (FWER)

• False Discovery rate (FDR)

• SPM results


•• RecapRecap & Introduction& Introduction


•• Single/multiple voxel Single/multiple voxel inferenceinference

•• FamilyFamily wisewise errorerror rate (FWER)rate (FWER)

•• False False DiscoveryDiscovery rate (FDR)rate (FDR)

•• SPM SPM resultsresults


Inference at a single voxelInference at a single voxelInference at a single voxel

−6 −4 −2 0 2 4 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

α = p(t>u|H)

NULL hypothesis, H: activation is zero

u=2t-distribution

p-value: probability of getting a value of t at least as extreme as u. If α is small we reject the null hypothesis H.

• Null Hypothesis H0

• Test statistic T– t observed realization of T

• α level– Acceptable false positive rate– Level α = P( T>uα | H0 )– Threshold uα controls false positive rate at level α

• P-value– Assessment of t assuming H0

– P( T > t | H0 )• Prob. of obtaining stat. as large

or larger in a new experiment– P(Data|Null) not P(Null|Data)

•• Null Hypothesis Null Hypothesis HH00

•• Test statistic Test statistic TT–– tt observed realization of observed realization of TT

•• αα levellevel–– Acceptable false positive rateAcceptable false positive rate–– Level Level αα = P( = P( TT>>uuαα | | HH00 ))–– Threshold Threshold uuαα controls false positive rate at level controls false positive rate at level αα

•• PP--valuevalue–– Assessment of Assessment of tt assuming assuming HH00

–– P( T > t | P( T > t | HH00 ))•• Prob. of obtaining stat. as largeProb. of obtaining stat. as large

or larger in a new experimentor larger in a new experiment–– P(Data|NullP(Data|Null) ) notnot P(Null|DataP(Null|Data))

Hypothesis TestingHypothesis TestingHypothesis Testing

uα

α

Null Distribution of T

t

P-val

Null Distribution of T

Inference at a single voxelInference at a single voxelInference at a single voxel

−6 −4 −2 0 2 4 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

α = p(t>u|H)

NULL hypothesis, H: activation is zero

u=2t-distribution

We can choose u to ensurea voxel-wise significance level of α.

This is called an ‘uncorrected’p-value, for reasons we’ll see later.

We can then plot a map of above threshold voxels.

• Don’t threshold, model the signal!– Signal location?

• Estimates and CI’s on(x,y,z) location

– Signal magnitude?• CI’s on % change

– Spatial extent?• Estimates and CI’s on activation volume• Robust to choice of cluster definition

• ...but this requires an explicit spatial model

•• DonDon’’t threshold, model the signal!t threshold, model the signal!–– Signal Signal locationlocation??

•• Estimates and CIEstimates and CI’’s ons on((x,y,zx,y,z) location) location

–– Signal Signal magnitudemagnitude??•• CICI’’s on % changes on % change

–– Spatial Spatial extentextent??•• Estimates and CIEstimates and CI’’s on activation volumes on activation volume•• Robust to choice of cluster definitionRobust to choice of cluster definition

•• ...but this requires an explicit spatial ...but this requires an explicit spatial modelmodel

What we’d likeWhat weWhat we’’d liked like

space

Loc.θ Ext.θ

Mag.θ

What we needWhat we needWhat we need

• Need an explicit spatial model• No routine spatial modeling methods

exist– High-dimensional mixture modeling

problem– Activations don’t look like Gaussian blobs– Need realistic shapes, sparse representation

• Some work by Hartvig et al., Penny et al.

•• Need an explicit spatial modelNeed an explicit spatial model•• No routine spatial modeling methods No routine spatial modeling methods

existexist–– HighHigh--dimensional mixture modeling dimensional mixture modeling

problemproblem–– Activations donActivations don’’t look like Gaussian blobst look like Gaussian blobs–– Need realistic shapes, sparse representationNeed realistic shapes, sparse representation

•• Some work by Some work by HartvigHartvig et al.et al., Penny , Penny et al.et al.

Real-life inference: What we getRealReal--life inference: What we getlife inference: What we get

• Signal location– Local maximum – no inference– Center-of-mass – no inference

• Sensitive to blob-defining-threshold

• Signal magnitude– Local maximum intensity – P-values (& CI’s)

• Spatial extent– Cluster volume – P-value, no CI’s

• Sensitive to blob-defining-threshold

•• Signal Signal locationlocation–– Local maximum Local maximum –– no inferenceno inference–– CenterCenter--ofof--mass mass –– no inferenceno inference

•• Sensitive to blobSensitive to blob--definingdefining--thresholdthreshold

•• Signal Signal magnitudemagnitude–– Local maximum intensity Local maximum intensity –– PP--values (& CIvalues (& CI’’s)s)

•• Spatial Spatial extentextent–– Cluster volume Cluster volume –– PP--value, no CIvalue, no CI’’ss

•• Sensitive to blobSensitive to blob--definingdefining--thresholdthreshold

Voxel-level InferenceVoxelVoxel--level Inferencelevel Inference

• Retain voxels above α-level threshold uα

• Gives best spatial specificity– The null hypothesis at a single voxel can be

rejected

•• Retain voxels above Retain voxels above αα--level threshold level threshold uuαα

•• Gives best spatial specificityGives best spatial specificity–– The null hypothesis at a single voxel can be The null hypothesis at a single voxel can be

rejectedrejected

Significant Voxels

space

uα

No significant Voxels

Cluster-level InferenceClusterCluster--level Inferencelevel Inference

• Two step-process–Define clusters by arbitrary threshold uclus

–Retain clusters larger than α-level threshold kα

•• Two stepTwo step--processprocess––Define clusters by arbitrary threshold Define clusters by arbitrary threshold uuclusclus

––Retain clusters larger than Retain clusters larger than αα--level threshold level threshold kkαα

Cluster not significant

uclus

space

Cluster significantkα kα

Cluster-level InferenceClusterCluster--level Inferencelevel Inference

• Typically better sensitivity• Worse spatial specificity

– The null hypothesis of entire cluster is rejected

– Only means that one or more of voxels in cluster active

•• Typically better sensitivityTypically better sensitivity•• Worse spatial specificityWorse spatial specificity

–– The null hypothesis of entire cluster is The null hypothesis of entire cluster is rejectedrejected

–– Only means that Only means that one or moreone or more of voxels in of voxels in cluster activecluster active

Cluster not significant

uclus

space

Cluster significantkα kα

Set-level InferenceSetSet--level Inferencelevel Inference

• Count number of blobs c– Minimum blob size k

• Worst spatial specificity– Only can reject global null hypothesis

•• Count number of blobs Count number of blobs cc–– Minimum blob size Minimum blob size kk

•• Worst spatial specificityWorst spatial specificity–– Only can reject global null hypothesisOnly can reject global null hypothesis

uclus

space

Here c = 1; only 1 cluster larger than kk k

Sensitivity and SpecificitySensitivity and SpecificitySensitivity and Specificity

H True (o) TN FP

H False (x) FN TP

Don’tReject

Reject

ACTION

TRUTH

Sens=10/10=100%Spec=7/10=70%

At u1

o o o o o o o x x x o o x x x o x x x xu1 u2

Eg. t-scoresfrom regionsthat truly do and do not activate

Sens=7/10=70%Spec=9/10=90%

At u2

Sensitivity = TP/(TP+FN) = β Specificity = TN/(TN+FP) = 1 - αFP = Type I error or ‘error’FN = Type II errorα = p-value/FP rate/error rate/significance levelβ = power

fMRI Multiple Comparisons ProblemfMRI Multiple Comparisons ProblemfMRI Multiple Comparisons Problem

• 4-Dimensional Data– 1,000 multivariate observations,

each with 100,000 elements– 100,000 time series, each

with 1,000 observations

• Massively UnivariateApproach– 100,000 hypothesis

tests

• Massive MCP!

•• 44--Dimensional DataDimensional Data–– 1,000 multivariate observations,1,000 multivariate observations,

each with 100,000 elementseach with 100,000 elements–– 100,000 time series, each 100,000 time series, each

with 1,000 observationswith 1,000 observations

•• Massively UnivariateMassively UnivariateApproachApproach–– 100,000 hypothesis100,000 hypothesis

teststests

•• Massive MCP!Massive MCP!

1,000

1

2

3. .

.

Inference for ImagesInference for ImagesInference for Images

Signal

Signal+Noise

Noise

11.3% 11.3% 12.5% 10.8% 11.5% 10.0% 10.7% 11.2% 10.2% 9.5%

Use of ‘uncorrected’ p-value, α=0.1

Percentage of Null Pixels that are False Positives

Using an ‘uncorrected’ p-value of 0.1 will lead us to conclude on average that 10% of voxels are active when they are not.

This is clearly undesirable : multiple comparison problem. To correct for this we can define a null hypothesis for images of statistics.

Multiple comparison problemMultiple comparison problemMultiple comparison problem

Multiple Comparisons ProblemMultiple Comparisons ProblemMultiple Comparisons Problem

• Which of 100,000 voxels are sig.?– α=0.05 ⇒ 5,000 false positive voxels

• Which of (random number, say) 100 clusters significant?– α=0.05 ⇒ 5 false positives clusters

•• Which of 100,000 voxels are sig.?Which of 100,000 voxels are sig.?–– αα=0.05 =0.05 ⇒⇒ 5,000 false positive voxels5,000 false positive voxels

•• Which ofWhich of (random number, say) (random number, say) 100 clusters 100 clusters significant?significant?–– αα=0.05 =0.05 ⇒⇒ 5 false positives clusters5 false positives clusters

t > 0.5 t > 1.5 t > 2.5 t > 3.5 t > 4.5 t > 5.5 t > 6.5

Assessing Statistic ImagesAssessing Statistic ImagesAssessing Statistic Images

Where’s the signal?WhereWhere’’s the signal?s the signal?

t > 0.5t > 3.5t > 5.5

High Threshold Med. Threshold Low Threshold

Good Specificity

Poor Power(risk of false negatives)

Poor Specificity(risk of false

positives)

Good Power

Multiple comparisons…Multiple comparisonsMultiple comparisons……

t59

Gaussian10mm FWHM(2mm pixels)

p = 0.05

• Threshold at p ?– expect (100 × p)% by chance

• Surprise ?– extreme voxel values

→voxel level inference– big suprathresholdclusters

→cluster level inference– many suprathresholdclusters

→set level inference

• Power & localisation→sensitivity→spatial specificity

•• Threshold at Threshold at p p ??–– expect (100 expect (100 ×× pp)% by )% by chancechance

•• Surprise Surprise ??–– extreme voxel valuesextreme voxel values

→→voxel level inferencevoxel level inference–– big big suprathresholdsuprathresholdclustersclusters

→→cluster level inferencecluster level inference–– many many suprathresholdsuprathresholdclustersclusters

→→set level inferenceset level inference

•• Power & localisationPower & localisation→→sensitivitysensitivity→→spatial specificityspatial specificity

Solutions forMultiple Comparison Problem

Solutions forSolutions forMultiple Comparison ProblemMultiple Comparison Problem

• A MCP Solution must control “False Positives”– How to measure multiple false positives?

• Familywise Error Rate (FWER)– Chance of any false positives– Controlled by Bonferroni, Random Field

Methods, non-parametric method (SnPM).

• False Discovery Rate (FDR)– Proportion of false positives among rejected

tests

•• A MCP Solution must control A MCP Solution must control ““False False PositivesPositives””–– How to measure multiple false positives?How to measure multiple false positives?

•• Familywise Error Rate (FWER)Familywise Error Rate (FWER)–– Chance of Chance of anyany false positivesfalse positives–– Controlled by Controlled by BonferroniBonferroni, Random Field , Random Field

Methods, nonMethods, non--parametric method (parametric method (SnPMSnPM).).

•• False Discovery Rate (FDR)False Discovery Rate (FDR)–– Proportion of false positives Proportion of false positives amongamong rejected rejected

teststests





• Family wise error rate (FWER)• Bonferroni correction/Random Field Theory• Non-parametric approach


• SPM results





•• FamilyFamily wisewise errorerror rate (FWER)rate (FWER)•• BonferroniBonferroni correction/correction/RandomRandom Field Field TheoryTheory•• NonNon--parametricparametric approachapproach




• Family of hypotheses– Hk k ∈ Ω = 1,…,K– HΩ = H1 ∩ H2 … ∩ Hk ∩ HK

•• FamilyFamily of hypothesesof hypotheses–– HHkk kk ∈∈ ΩΩ = 1,= 1,……,,KK–– HHΩΩ = = HH11 ∩∩ HH22 …… ∩∩ HHkk ∩∩ HHKK

Family-wise Null HypothesisFamilyFamily--wise Null Hypothesiswise Null Hypothesis

FAMILY-WISE NULL HYPOTHESIS:Activation is zero everywhere

If we reject a voxel null hypothesis at any voxel, we reject the family-wise null hypothesis

A FP anywhere gives a Family Wise Error (FWE)

Family-wise error rate = ‘corrected’ p-value

FWE MCP Solutions: Controlling FWE w/ MaxFWE MCP Solutions: FWE MCP Solutions:

Controlling FWE w/ MaxControlling FWE w/ Max

• FWE & distribution of maximumFWE = P(FWE)

= P( ∪i Ti ≥ u | Ho)= P( maxi Ti ≥ u | Ho)

• 100(1-α)%ile of max distn controls FWEFWE = P( maxi Ti ≥ uα | Ho) = α

– whereuα = F-1

max (1-α)

.

•• FWE & distribution of maximumFWE & distribution of maximumFWEFWE = P(FWE)= P(FWE)

= P( = P( ∪∪ii TTii ≥≥ uu | | HHoo))= P( max= P( maxii TTi i ≥≥ uu | | HHoo))

•• 100(1100(1--αα)%ile of )%ile of max max distdistnn controls FWEcontrols FWEFWE = P( maxFWE = P( maxii TTi i ≥≥ uuαα | | HHoo) ) = = αα

–– wherewhereuuαα = = FF--11

max max (1(1--αα))

..

uα

α

Example: Experiment with 100,000 « voxels » and 40 d.f.

type I error α=0.05 (5% risk) ⇒ tα = 1.68100,000 t values ⇒ 5000 t values > 1.68

just by chance !

Familywise Error I test, PFWE:find threshold tα such that, in a family of 100,000 tstatistics, only 5% probability of one or more tvalues above that threshold

ExampleExample: : ExperimentExperiment withwith 100,000 100,000 «« voxelsvoxels »» and 40 d.f.and 40 d.f.

type I type I errorerror αα=0.05 (5% =0.05 (5% riskrisk) ) ⇒⇒ ttαα = 1.68= 1.68100,000 100,000 t t values values ⇒⇒ 5000 5000 tt values > 1.68values > 1.68

justjust by chance !by chance !

FamilywiseFamilywise ErrorError I test, I test, PPFWEFWE::findfind thresholdthreshold ttαα suchsuch thatthat, in a , in a familyfamily of 100,000 of 100,000 ttstatisticsstatistics, , onlyonly 5% 5% probabilityprobability of one or more of one or more ttvalues values aboveabove thatthat thresholdthreshold

Multiple comparison problem…Multiple comparison problemMultiple comparison problem……

Bonferroni correction:simple method to find the new threshold

Random field theory:more accurate for functional imaging

Given • a family of N independent voxels and • a voxel-wise error rate v

The probability that all tests are below the threshold, i.e. that Ho is true : (1- v)N

The Family-Wise Error rate (FWE) or ‘corrected’ errorrate α is

α = 1 – (1-v)N

~ Nv (for small v)

Therefore, to ensure a particular FWE we choosev = α/N

A Bonferroni correction is appropriate for independent tests.

Given Given •• a family of a family of NN independent voxels and independent voxels and •• a voxela voxel--wise error rate wise error rate v

The probability that all tests are below the threshold, i.e. that Ho is true : (1- v)N

The FamilyThe Family--Wise Error rate (FWE) or Wise Error rate (FWE) or ‘‘correctedcorrected’’ errorerrorrate rate αα isis

αα = 1 = 1 –– (1(1--v)v)NN

~ Nv (for small v)

Therefore, to ensure a particular FWE we choosev = α/N

A Bonferroni correction is appropriate for independent tests.

The “Bonferroni” correction…The The ““BonferroniBonferroni”” correctioncorrection……

Experiment with N = 100,000 « voxels » and 40 d.f.– v = unknown corrected probability threshold, – find v such that family-wise error rate α = 0.05

Bonferroni correction:– probability that all tests are below the threshold,– Use v = α / N– Here v=0.05/100000=0.0000005

⇒ threshold t = 5.77Interpretation:

Bonferroni procedure gives a corrected p value, i.e. for a t statistics = 5.77, – uncorrectd p value = 0.0000005– corrected p value = 0.05

ExperimentExperiment withwith N = 100,000 N = 100,000 «« voxelsvoxels »» and 40 d.f.and 40 d.f.– v = unknown corrected probability threshold, – find v such that family-wise error rate α = 0.05

BonferroniBonferroni correction:correction:– probability that all tests are below the threshold,– Use v = α / N– Here v=0.05/100000=0.0000005

⇒ threshold t = 5.77InterpretationInterpretation::

Bonferroni procedure gives a corrected p value, i.e. for a t statistics = 5.77, – uncorrectd p value = 0.0000005– corrected p value = 0.05

The “Bonferroni” correction…The The ““BonferroniBonferroni”” correctioncorrection……

Use of ‘uncorrected’ p-value, α=0.1

FWE

Use of ‘corrected’ p-value, α=0.1

“Bonferroni” correction & independent observations““BonferroniBonferroni”” correction & correction & independent observationsindependent observations

100 by 100 voxels, with a z value.10000 independent measuresFix the PFWE = 0.05, z threshold ?

100 by 100 voxels, with a z value.100 independent measuresFix the PFWE = 0.05, z threshold ?

v=α/ni where ni is the number of independent observations.

Bonferroni: v = 0.05 / 10000 = 0.000005

⇒ threshold z = 4.42

Bonferroni: v = 0.05 / 100 = 0.0005

⇒ threshold z = 3.29

“Bonferroni” correction & smoothed observations

““BonferroniBonferroni”” correction & correction & smoothed observationssmoothed observations

100 by 100 voxels, with a zvalue.10000 independent measuresFix the PFWE = 0.05, z threshold ?Bonferroni:

v = 0.05/10000 = 0.000005 ⇒ threshold z = 4.42

100 by 100 voxels, with a zvalue.How many independentmeasures ?

Fix the PFWE = 0.05, z threshold ?Bonferroni ?

Random Field TheoryRandom Field TheoryRandom Field Theory

• Consider a statistic image as a lattice representation of a continuous random field

• Use results from continuous random field theory

•• Consider a statistic image as a lattice Consider a statistic image as a lattice representation of a continuous random fieldrepresentation of a continuous random field

•• Use results from continuous random field theoryUse results from continuous random field theory

Latticerepresentation

Euler Characteristic (EC)Euler Characteristic (EC)Euler Characteristic (EC)

Topological measure– threshold an image at u– excursion set Au

− χ(Αu) = # blobs - # holes- At high u, χ(Αu) = # blobs

Reject HΩ if Euler characteristic non-zero

α ≈ Pr(χ(Αu) > 0 )

Expected Euler chararcteristic≈ p–value

(at high u)

α ≈ E[χ(Αu)]

Topological measureTopological measure–– threshold an image at threshold an image at uu–– excursion set excursion set Auu

−− χχ(ΑΑuu) == # blobs # blobs -- # holes# holes-- At high u,At high u, χχ(ΑΑuu) == # blobs# blobs

Reject HReject HΩΩ if if Euler Euler characteristic noncharacteristic non--zerozero

αα ≈≈ Pr(Pr(χχ(ΑΑuu) > 0 ) > 0 )

Expected Euler Expected Euler chararcteristicchararcteristic≈≈ pp––valuevalue

(at high u)(at high u)

αα ≈≈ EE[[χχ(ΑΑuu)]]

Euler characteristic…Euler characteristicEuler characteristic……

Euler characteristic (EC) ≈ # blobs in athresholded image.

(True only for high threshold)

EC = function of • threshold used• number of resels

where resels (« resolutionelements »)~ number of independent obsevations

⇒ E[EC] ≈ PFWE


Threshold z-mapat 2.75

EC = 1

Threshold z-mapat 2.50

EC = 3


Euler characteristic (EC) ≈ # blobs in athresholded image.

(True only for high threshold)

For a threshold ztat 2.50, E[EC] = 1.9at 2.75, E[EC] = 1.1 instead of 3 and 1 as in example

EC = function of • threshold used• number of resels

where resels (« resolutionelements »)~ number of independent obsevations

⇒ E[EC] ≈ PFWE

E[χ(Αu)] ≈ λ(Ω) √ |Λ| (u 2 -1) exp(-u 2/2) / (2π)2

– Ω → large search region Ω ⊂ R3

– λ(Ω) → volume– √|Λ| → smoothness– Au → excursion set Au = x ∈ R3 : Z(x) > u– Z(x) → Gaussian random field x ∈ R3

+ Multivariate Normal Finite Dimensional distributions+ continuous+ strictly stationary+ marginal N(0,1)

+ continuously differentiable

+ twice differentiable at 0+ Gaussian ACF

(at least near local maxima)

EE[[χχ(ΑΑuu)] ] ≈≈ λλ((ΩΩ)) √√ ||ΛΛ|| (u u 2 2 --11) expexp(--u u 22/2/2) / (22ππ)22

–– ΩΩ →→ largelarge search regionsearch region ΩΩ ⊂⊂ RR3 3

–– λλ((Ω)Ω) →→ volumevolume–– √√||ΛΛ|| →→ smoothnesssmoothness–– AAuu →→ excursion setexcursion set AAuu = x ∈ RR33 : Z(x) > u– Z(x) →→ Gaussian Gaussian random fieldrandom field x ∈ RR33

+ + Multivariate Normal Finite Dimensional distributionsMultivariate Normal Finite Dimensional distributions

+ + continuouscontinuous

+ + strictly stationarystrictly stationary

+ + marginal Nmarginal N((0,10,1))

++ continuously differentiablecontinuously differentiable

++ twice differentiable at 0twice differentiable at 0

++ Gaussian Gaussian ACFACF

(at least near local maxima)(at least near local maxima)

Expected Euler characteristic…Expected Euler characteristicExpected Euler characteristic……

Au

Ω

• General form for expected Euler characteristic• χ2, F, & t fields • restricted search regions

α = Σ Rd (Ω) ρd (u)

•• General form for expected Euler characteristicGeneral form for expected Euler characteristic•• χχ22, , FF, & , & tt fields fields •• restricted search regionsrestricted search regions

αα = = ΣΣ RRd d ((ΩΩ)) ρρd d ((uu))

Unified TheoryUnified TheoryUnified Theory

Rd (Ω), RESEL count depends on :

• the search region– how big, how smooth,

what shape ?

ρd (υ): EC density depends on :

• type of field (eg. Gaussian, t)• the threshold, u.

Worsley et al. (1996), HBM

Au

Ω

• General form for expected Euler characteristic• χ2, F, & t fields • restricted search regions

α = Σ Rd (Ω) ρd (u)

•• General form for expected Euler characteristicGeneral form for expected Euler characteristic•• χχ22, , FF, & , & tt fields fields •• restricted search regionsrestricted search regions

αα = = ΣΣ RRd d ((ΩΩ)) ρρd d ((uu))

Unified TheoryUnified TheoryUnified Theory

Rd (Ω), d-dimensional Minkowski functional of Ω

• R0(Ω)=χ(Ω) Euler characteristic of Ω

• R1(Ω)=resel diameter

• R2(Ω)=resel surface area

• R3(Ω)=resel volume

ρd (u), d-dimensional EC density :E.g. Gaussian RF:

ρ0(u) = 1- Φ(u)

ρ1(u) = (4 ln2)1/2 exp(-u2/2) / (2π)

ρ2(u) = (4 ln2) exp(-u2/2) / (2π)3/2

ρ3(u) = (4 ln2)3/2 (u2 -1) exp(-u2/2) / (2π)2

ρ4(u) = (4 ln2)2 (u3 -3u) exp(-u2/2) / (2π)5/2

Au

Ω

Worsley et al. (1996), HBM

Estimated component fieldsEstimated component fieldsEstimated component fields

data matrix

desi

gn

m

atr

ix

parameters errors+ ?= × ?voxelsvoxels

scansscans

estimate

β

residuals

estimatedcomponent

fields

parameterestimates

estimated variance÷=

Each row isan estimatedcomponent field

• RESELS– Resolution Elements– 1 RESEL = FWHMx × FWHMy × FWHMz– RESEL Count R

• R = λ(Ω) √ |Λ| = (4log2)3/2 λ(Ω) / ( FWHMx × FWHMy × FWHMz ) • Volume of search region in units of smoothness

– Eg: 10 voxels, 2.5 FWHM 4 RESELS

• Beware RESEL misinterpretation– RESEL are not “number of independent ‘things’ in the image”

• See Nichols & Hayasaka, 2003, Stat. Meth. in Med. Res..

•• RESELSRESELS–– ResResolution olution ElElementsements–– 1 RESEL = 1 RESEL = FWHMFWHMxx × × FWHMFWHMyy × × FWHMFWHMzz–– RESEL Count RESEL Count RR

•• RR = = λλ((ΩΩ)) √√ ||ΛΛ|| = (4log2)= (4log2)3/2 3/2 λλ((ΩΩ)) / ( / ( FWHMFWHMxx × × FWHMFWHMyy × × FWHMFWHMzz ) ) •• Volume of search region in units of smoothnessVolume of search region in units of smoothness

–– EgEg: : 10 voxels10 voxels, 2.5 FWHM , 2.5 FWHM 4 RESELS4 RESELS

•• Beware RESEL misinterpretationBeware RESEL misinterpretation–– RESEL RESEL are not are not ““number of independent number of independent ‘‘thingsthings’’ in the imagein the image””

•• See Nichols & Hayasaka, 2003, Stat. See Nichols & Hayasaka, 2003, Stat. MethMeth. in Med. Res.. in Med. Res...

Random Field TheorySmoothness Parameterization

Random Field TheoryRandom Field TheorySmoothness ParameterizationSmoothness Parameterization

1 2 3 4

2 4 6 8 101 3 5 7 9

[ ] [ ] [ ][ ] [ ] [ ][ ] [ ] [ ] ⎟

⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

=Λ

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

∂∂

ze

ze

ye

ze

xe

ze

ye

ye

ye

xe

ze

xe

ye

xe

xe

var,cov,cov,covvar,cov,cov,covvar

Smoothness, PRF, resels...Smoothness, PRF, Smoothness, PRF, reselsresels......

• Smoothness √|Λ|– variance-covariance matrix of

partial derivatives (possibly location dependent)

• Point Response Function PRF

• Full Width at Half Maximum FWHM. Approximate the peak of the Covariance function with a Gaussian

•• SmoothnessSmoothness √√||ΛΛ||–– variancevariance--covariance matrix of covariance matrix of

partial derivatives partial derivatives (possibly location (possibly location dependent)dependent)

•• Point Response Function Point Response Function PRFPRF

•• Full Width at Half Maximum Full Width at Half Maximum FWHM.FWHM. Approximate the peak of the Covariance function with a Gaussian

• Gaussian PRF– Σ – kernel var/cov matrix– ACF 2Σ– Λ = (2Σ)-1

⇒FWHM f = σ √(8ln(2))fx 0 0

– Σ = 0 fy 0 10 0 fz 8ln(2)

ignoring covariances

⇒ √|Λ| = (4ln(2))3/2 / (fx × fy × fz)

• Resolution Element (RESEL)– Resel dimensions (fx × fy × fz)– R3(Ω) = λ(Ω) / (fx × fy × fz)

if strictly stationary

E[χ(Αu)] = R3(Ω) (4ln(2))3/2 (u 2 -1) exp(-u 2/2) / (2π)2

≈ R3(Ω) (1 – Φ(u)) for high thresholds u

•• Gaussian Gaussian PRFPRF–– ΣΣ –– kernel kernel var/covvar/cov matrixmatrix–– ACFACF 22ΣΣ–– ΛΛ = (2= (2ΣΣ))--11

⇒⇒FWHMFWHM f = f = σσ √√(8ln(2))(8ln(2))ffxx 0 0

–– ΣΣ = = 00 ffyy 0 10 0 ffzz 8ln(2)

ignoring ignoring covariancescovariances

⇒⇒ √√||ΛΛ|| = (4ln(2))= (4ln(2))3/23/2 / (/ (ffxx ×× ffyy ×× ffzz))

•• ResResolution olution ElElement (ement (RESELRESEL))–– ReselResel dimensionsdimensions ((ffxx ×× ffyy ×× ffzz))–– RR33((ΩΩ) = ) = λλ((ΩΩ)) / (/ (ffxx ×× ffyy ×× ffzz))

if strictly stationaryif strictly stationary

EE[[χχ(ΑΑuu)]] = R= R33((ΩΩ) (4ln(2))) (4ln(2))3/23/2 ((u u 2 2 --1) exp(1) exp(--u u 22/2) / (2/2) / (2ππ))22

≈≈ RR33((ΩΩ) ) ((1 1 –– ΦΦ(u)(u))) for high thresholds ufor high thresholds u

Λ

RFT AssumptionsRFT AssumptionsRFT Assumptions

• Model fit & assumptions– valid distributional

results

• Multivariate normality– of component images

• Covariance function of component images must be

- Can be nonstationary

- Twice differentiable

•• Model fit & assumptionsModel fit & assumptions–– valid distributional valid distributional

resultsresults

•• Multivariate normalityMultivariate normality–– of of componentcomponent imagesimages

•• Covariance function of Covariance function of componentcomponent images must images must bebe

-- Can be Can be nonstationarynonstationary

-- Twice differentiableTwice differentiable

Smoothnesssmoothness » voxel size

lattice approximationsmoothness estimation

practicallyFWHM ≥ 3 × VoxDim

otherwiseconservative

“Typical” applied smoothing:Single Subj fMRI: 6mm

PET: 12mmMulti Subj fMRI: 8-12mm

PET: 16mm

Level of smoothing should actually depend on what

you’re looking for…

SmoothnessSmoothnesssmoothness smoothness »» voxel sizevoxel size

lattice approximationlattice approximationsmoothness estimationsmoothness estimation

practicallypracticallyFWHMFWHM ≥≥ 3 3 ×× VoxDimVoxDim

otherwiseotherwiseconservativeconservative

““TypicalTypical”” applied smoothing:applied smoothing:Single Single SubjSubj fMRI: 6mmfMRI: 6mm

PET: 12mmPET: 12mmMulti Multi SubjSubj fMRI: 8fMRI: 8--12mm12mm

PET: 16mm PET: 16mm

Level of smoothing should Level of smoothing should actually depend on what actually depend on what

youyou’’re looking forre looking for……

Random Field IntuitionRandom Field IntuitionRandom Field Intuition

• Corrected P-value for voxel value tPc = P(max T > t)

≈ E(χt)≈ λ(Ω) |Λ|1/2 t2 exp(-t2/2)

• Statistic value t increases– Pc decreases (but only for large t)

• Search volume increases (bigger Ω)

– Pc increases (more severe MCP)• Smoothness increases (roughness |Λ|1/2 decreases)

– Pc decreases (less severe MCP)

•• Corrected PCorrected P--value for voxel value value for voxel value ttPPcc = = P(maxP(max TT > > tt))

≈≈ E(E(χχtt))≈≈ λλ((ΩΩ) |) |ΛΛ||1/21/2 tt2 2 exp(exp(--tt22/2)/2)

•• Statistic value Statistic value tt increasesincreases–– PPcc decreases (but only for large decreases (but only for large tt))

•• Search volume increases Search volume increases (bigger (bigger ΩΩ))

–– PPcc increases (more severe MCP)increases (more severe MCP)•• Smoothness increases Smoothness increases (roughness (roughness ||ΛΛ||1/21/2 decreases)decreases)

–– PPcc decreases (less severe MCP)decreases (less severe MCP)

Small Volume CorrectionSmall Volume CorrectionSmall Volume Correction

SVC = correction for multiple comparison in a user’s defined volume ‘of interest’.

SVC = correction for multiple comparison in a SVC = correction for multiple comparison in a useruser’’s defined volume s defined volume ‘‘of interestof interest’’..

Shape and size of volume becomeimportant for small or oddly shaped volume !

Example of SVC (900 voxels)• compact volume: samples

from maximum 16 resels• spread volume: sample

from up to 36 resels⇒ threshold higher for

spread volume thancompact volume.

Resel Counts for Brain StructuresReselResel Counts for Brain StructuresCounts for Brain Structures

FWHM=20mm(1) Threshold depends on Search Volume(2) Surface area makes a large contribution

SummarySummarySummary

• We should correct for multiple comparisons– We can use Random Field Theory (RFT) or other methods

• RFT requires– a good lattice approximation to underlying multivariate

Gaussian fields,

– that these fields are continuous with a twice differentiable correlation function

• To a first approximation, RFT is a Bonferroni correction using RESELS.

• We only need to correct for the volume of interest.

• Depending on nature of signal we can trade-off anatomical specificity for signal sensitivity with the use of cluster-level inference.

• We should correct for multiple comparisons– We can use Random Field Theory (RFT) or other methods

• RFT requires– a good lattice approximation to underlying multivariate

Gaussian fields,

– that these fields are continuous with a twice differentiable correlation function

• To a first approximation, RFT is a Bonferroni correction using RESELS.

• We only need to correct for the volume of interest.

• Depending on nature of signal we can trade-off anatomical specificity for signal sensitivity with the use of cluster-level inference.





• Family wise error rate (FWER)• Bonferroni correction/Random Field Theory• Non-parametric approach


• SPM results





•• FamilyFamily wisewise errorerror rate (FWER)rate (FWER)•• BonferroniBonferroni correction/correction/RandomRandom Field Field TheoryTheory•• NonNon--parametricparametric approachapproach




NonparametricPermutation TestNonparametric

Permutation Test

• Parametric methods– Assume distribution of

statistic under nullhypothesis

• Nonparametric methods– Use data to find

distribution of statisticunder null hypothesis

– Any statistic!


statistic under nullhypothesis


distribution of statisticunder null hypothesis

– Any statistic!

5%

Parametric Null Distribution

5%

Nonparametric Null Distribution

Permutation Test : Toy ExamplePermutation Test : Toy Example

• Data from V1 voxel in visual stim. experimentA: Active, flashing checkerboard B: Baseline, fixation6 blocks, ABABAB Just consider block averages...

• Null hypothesis Ho– No experimental effect, A & B labels arbitrary

• Statistic– Mean difference

• Data from V1 voxel in visual stim. experimentA: Active, flashing checkerboard B: Baseline, fixation6 blocks, ABABAB Just consider block averages...

• Null hypothesis Ho– No experimental effect, A & B labels arbitrary

• Statistic– Mean difference

96.0696.0699.7699.7687.8387.8399.9399.9390.4890.48103.00103.00

BBAABBAABBAA


• Under Ho– Consider all equivalent relabelings

•• Under Under HHoo

–– Consider all equivalent Consider all equivalent relabelingsrelabelings

BBBAAABBBAAABABABABABABAABBBAAABBBAAABAABBABAABB

BBABAABBABAABABAABBABAABABBABAABBABAAABBBAAABBBA

BBAABABBAABABAABBABAABBAABBAABABBAABAABBABAABBAB

BBAAABBBAAABBAABABBAABABABABBAABABBAAABABBAABABB

BABBAABABBAABAAABBBAAABBABABABABABABAAABBBAAABBB


• Under Ho– Consider all equivalent relabelings– Compute all possible statistic values


–– Consider all equivalent Consider all equivalent relabelingsrelabelings–– Compute all possible statistic valuesCompute all possible statistic values

BBBAAA BBBAAA --4.824.82BABABA BABABA --9.459.45ABBBAA 1.48ABBBAA 1.48ABAABB 6.86ABAABB 6.86

BBABAA 3.25BBABAA 3.25BABAAB BABAAB --6.976.97ABBABA ABBABA --1.101.10AABBBA AABBBA --3.153.15

BBAABA 0.67BBAABA 0.67BAABBA BAABBA --1.381.38ABBAAB 1.38ABBAAB 1.38AABBAB AABBAB --0.670.67

BBAAAB 3.15BBAAAB 3.15BAABAB 1.10BAABAB 1.10ABABBA 6.97ABABBA 6.97AABABB AABABB --3.253.25

BABBAA BABBAA --6.866.86BAAABB BAAABB --1.481.48ABABAB 9.45ABABAB 9.45AAABBB 4.82AAABBB 4.82


• Under Ho– Consider all equivalent relabelings– Compute all possible statistic values– Find 95%ile of permutation distribution


–– Consider all equivalent Consider all equivalent relabelingsrelabelings–– Compute all possible statistic valuesCompute all possible statistic values–– Find 95%ile of permutation distributionFind 95%ile of permutation distribution





BABBAA BABBAA --6.866.86BAAABB BAAABB --1.481.48ABABAB 9.45ABABAB 9.45AAABBB 4.82AAABBB 4.82




0 4 8-4-8




–– Consider all equivalent Consider all equivalent relabelingsrelabelings–– Compute all possible statistic valuesCompute all possible statistic values–– Find 95%ile of permutation distributionFind 95%ile of permutation distribution





BABBAA BABBAA --6.866.86BAAABB BAAABB --1.481.48ABABABABABAB 9.459.45AAABBB 4.82AAABBB 4.82

Controlling FWER: Permutation TestControlling FWER: Permutation Test


max statistic under nullhypothesis


distribution of max statisticunder null hypothesis

– Again, any max statistic!


max statistic under nullhypothesis


distribution of max statisticunder null hypothesis

– Again, any max statistic!

5%

Parametric Null Max Distribution

5%

Nonparametric Null Max Distribution

Permutation Test & ExchangeabilityPermutation Test & Exchangeability

• Exchangeability is fundamental– Def: Distribution of the data unperturbed by

permutation– Under H0, exchangeability justifies permuting data– Allows us to build permutation distribution

• Subjects are exchangeable– Under Ho, each subject’s A/B labels can be flipped

• Are fMRI scans exchangeable under Ho?– If no signal, can we permute over time?

• Exchangeability is fundamental– Def: Distribution of the data unperturbed by

permutation– Under H0, exchangeability justifies permuting data– Allows us to build permutation distribution

• Subjects are exchangeable– Under Ho, each subject’s A/B labels can be flipped

• Are fMRI scans exchangeable under Ho?– If no signal, can we permute over time?

Permutation Test & ExchangeabilityPermutation Test & Exchangeability

• fMRI scans are not exchangeable– Permuting disrupts order, temporal autocorrelation

• Intrasubject fMRI permutation test– Must decorrelate data, model before permuting– What is correlation structure?

• Usually must use parametric model of correlation

– E.g. Use wavelets to decorrelate• Bullmore et al 2001, HBM 12:61-78

• Intersubject fMRI permutation test– Create difference image for each subject– For each permutation, flip sign of some subjects

•• fMRI scans are not exchangeablefMRI scans are not exchangeable–– Permuting disrupts order, temporal autocorrelationPermuting disrupts order, temporal autocorrelation

•• IntraIntrasubject fMRI permutation testsubject fMRI permutation test–– Must decorrelate data, model before permutingMust decorrelate data, model before permuting–– What is correlation structure?What is correlation structure?

•• Usually must use parametric model of correlationUsually must use parametric model of correlation

–– E.g. Use wavelets to E.g. Use wavelets to decorrelatedecorrelate•• BullmoreBullmore et al 2001, HBM 12:61et al 2001, HBM 12:61--7878

•• InterIntersubject fMRI permutation testsubject fMRI permutation test–– Create difference image for each subjectCreate difference image for each subject–– For each permutation, flip sign of some subjectsFor each permutation, flip sign of some subjects

Permutation Test : ExamplePermutation Test : ExamplePermutation Test : Example

• fMRI Study of Working Memory – 12 subjects, block design Marshuetz et al (2000)

– Item Recognition• Active:View five letters, 2s pause,

view probe letter, respond• Baseline: View XXXXX, 2s pause,

view Y or N, respond

• Second Level RFX– Difference image, A-B constructed

for each subject– One sample, smoothed variance t test

• fMRI Study of Working Memory – 12 subjects, block design Marshuetz et al (2000)

– Item Recognition• Active:View five letters, 2s pause,

view probe letter, respond• Baseline: View XXXXX, 2s pause,

view Y or N, respond

• Second Level RFX– Difference image, A-B constructed

for each subject– One sample, smoothed variance t test

...

D

yes

...

UBKDA

Active

...

N

no

...

XXXXX

Baseline

Permutation Test : ExamplePermutation Test : Example

• Permute!– 212 = 4,096 ways to flip 12 A/B labels– For each, note maximum of t image.

•• Permute!Permute!–– 221212 = 4,096 ways to flip 12 A/B labels= 4,096 ways to flip 12 A/B labels–– For each, note maximum of For each, note maximum of t t imageimage..

Permutation DistributionMaximum t

Maximum Intensity Projection Thresholded t

t11 Statistic, RF & Bonf. Thresholdt11 Statistic, Nonparametric Threshold

uRF = 9.87uBonf = 9.805 sig. vox.

uPerm = 7.67 58 sig. vox.

Test Level vs. t11 Threshold

•Compare with Bonferroniα = 0.05/110,776

•Compare with parametric RFT110,776 2×2×2mm voxels5.1×5.8×6.9mm FWHM

smoothness462.9 RESELs

Does this Generalize?RFT vs Bonf. vs Perm.

Does this Generalize?RFT vs Bonf. vs Perm.

t Threshold (0.05 Corrected)

df RF Bonf PermVerbal Fluency 4 4701.32 42.59 10.14Location Switching 9 11.17 9.07 5.83Task Switching 9 10.79 10.35 5.10Faces: Main Effect 11 10.43 9.07 7.92Faces: Interaction 11 10.70 9.07 8.26Item Recognition 11 9.87 9.80 7.67Visual Motion 11 11.07 8.92 8.40Emotional Pictures 12 8.48 8.41 7.15Pain: Warning 22 5.93 6.05 4.99Pain: Anticipation 22 5.87 6.05 5.05

RFT vs Bonf. vs Perm.RFT vs Bonf. vs Perm.

No. Significant Voxels (0.05 Corrected)

t df RF Bonf Perm

Verbal Fluency 4 0 0 0Location Switching 9 0 0 158Task Switching 9 4 6 2241Faces: Main Effect 11 127 371 917Faces: Interaction 11 0 0 0Item Recognition 11 5 5 58Visual Motion 11 626 1260 1480Emotional Pictures 12 0 0 0Pain: Warning 22 127 116 221Pain: Anticipation 22 74 55 182

Performance SummaryPerformance SummaryPerformance Summary

• Bonferroni– Not adaptive to smoothness– Not so conservative for low

smoothness

• Random Field– Adaptive– Conservative for low smoothness & df

• Permutation– Adaptive (Exact)

•• BonferroniBonferroni–– Not adaptive to smoothnessNot adaptive to smoothness–– Not so conservative for low Not so conservative for low

smoothnesssmoothness

•• Random FieldRandom Field–– AdaptiveAdaptive–– Conservative for low smoothness & Conservative for low smoothness & dfdf

•• PermutationPermutation–– Adaptive (Exact)Adaptive (Exact)

“Old” Conclusions“Old” Conclusions

• t random field results conservative for– Low df & smoothness– 9 df & ≤12 voxel FWHM; 19 df & < 10

voxel FWHM

• Bonferroni surprisingly satisfactory for low smoothness

• Nonparametric methods perform well overall

• t random field results conservative for– Low df & smoothness– 9 df & ≤12 voxel FWHM; 19 df & < 10

voxel FWHM

• Bonferroni surprisingly satisfactory for low smoothness

• Nonparametric methods perform well overall







• SPM results









False Discovery RateFalse Discovery RateFalse Discovery Rate

H True (o) TN FP

H False (x) FN TP

Don’tReject

Reject

ACTION

TRUTH

FDR=3/17=18%α=3/10=30%

At u1

o o o o o o o x x x o o x x x o x x x x x x x x

u1 u2

Eg. t-scoresfrom regionsthat truly do and do not activate

FDR=1/12=8%α=1/10=10%

At u2

FDR = FP/(FP+TP)

α = FP/(FP+TN)

False Discovery RateIllustration:

False Discovery RateFalse Discovery RateIllustration:Illustration:

Signal

Signal+Noise

Noise

11.3% 11.3% 12.5% 10.8% 11.5% 10.0% 10.7% 11.2% 10.2% 9.5%

Control of Per Comparison Rate at 10%

Percentage of Null Pixels that are False Positives

FWE

Control of Familywise Error Rate at 10%

Occurrence of Familywise Error

6.7% 10.4% 14.9% 9.3% 16.2% 13.8% 14.0% 10.5% 12.2% 8.7%

Control of False Discovery Rate at 10%

Percentage of Activated Pixels that are False Positives

Benjamini & Hochberg ProcedureBenjaminiBenjamini & Hochberg Procedure& Hochberg Procedure

• Select desired limit q on E(FDR)• Order p-values, p(1) ≤ p(2) ≤ ... ≤ p(V)

• Let r be largest i such that

• Reject all hypotheses corresponding top(1), ... , p(r).

•• Select desired limit Select desired limit qq on E(FDR)on E(FDR)•• Order pOrder p--values, values, pp(1)(1) ≤≤ pp(2)(2) ≤≤ ... ... ≤≤ pp((VV))

•• Let Let rr be largest be largest ii such thatsuch that

•• Reject all hypotheses Reject all hypotheses corresponding tocorresponding topp(1)(1), ... , , ... , pp((rr))..

p(i) ≤ i/V*q

p(i)

i/Vi/V × q/c(V)

p-va

lue

0 1

01

JRSS-B (1995) 57:289-300

Benjamini & Hochberg:Varying Signal Extent

BenjaminiBenjamini & Hochberg:& Hochberg:Varying Signal ExtentVarying Signal Extent

Signal Intensity 3.0 Signal Extent 1.0 Noise Smoothness 3.0

p = z =

1Signal Intensity 3.0 Signal Extent 2.0 Noise Smoothness 3.0

p = z =

2




p = z =

3




p = 0.000252 z = 3.48

4




p = 0.001628 z = 2.94

5



Signal Intensity 3.0 Signal Extent16.5 Noise Smoothness 3.0

p = 0.007157 z = 2.45

6



Signal Intensity 3.0 Signal Extent25.0 Noise Smoothness 3.0

p = 0.019274 z = 2.07

7



Benjamini & Hochberg: PropertiesBenjaminiBenjamini & Hochberg: & Hochberg: PropertiesProperties

• Adaptive– Larger the signal, the lower the threshold– Larger the signal, the more false positives

• False positives constant as fraction of rejected tests

• Not a problem with imaging’s sparse signals

• Smoothness OK– Smoothing introduces positive correlations

•• AdaptiveAdaptive–– Larger the signal, the lower the thresholdLarger the signal, the lower the threshold–– Larger the signal, the more false positivesLarger the signal, the more false positives

•• False positives constant as fraction of False positives constant as fraction of rejected testsrejected tests

•• Not a problem with imagingNot a problem with imaging’’s sparse s sparse signalssignals

•• Smoothness OKSmoothness OK–– Smoothing introduces positive correlationsSmoothing introduces positive correlations







• SPM results









SPM position

u

SPM intensity

h

L1 L2

: significant at the set level

: significant at the cluster level

: significant at the voxel level

L1 > spatial extent threshold

L2 < spatial extent threshold

Regional specificity

Sensitivity

The intensity of a voxel

Test based on

The spatial extent above u or the spatial extent and the

maximum peak height

The number of clusters above u with size greater

than n

The sum of square of the SPM or a MANOVA

Parameters set by the user

• Low pass filter

• Low pass filter• intensity threshold u

• Low pass filter• intensity thres. u• spatial threshold n

• Low pass filter

Summary: Levels of inference & powerSummary: Summary: Levels of inference & powerLevels of inference & power

Levels of inference…Levels of inferenceLevels of inference……

ParametersParametersu u -- 3.093.09k k -- 12 voxels12 voxelsS S -- 32323 3 voxelsvoxelsFWHMFWHM -- 4.7 voxels4.7 voxelsD D -- 33

n=82n=82

n=32n=32

n=12n=12

omnibusomnibusP(cP(c≥≥7 | n 7 | n ≥≥ 0, t 0, t ≥≥ 3.09) = 0.0313.09) = 0.031

setset--levellevelP(c P(c ≥≥ 3 | n 3 | n ≥≥ 12, t 12, t ≥≥ 3.09) = 0.0193.09) = 0.019

clustercluster--levellevelP(c P(c ≥≥ 1 | n 1 | n ≥≥ 82, t 82, t ≥≥ 3.09) = 0.029 3.09) = 0.029 (corrected)(corrected)P(n P(n ≥≥ 82 | t 82 | t ≥≥ 3.09) = 0.019 3.09) = 0.019 (uncorrected)(uncorrected)

voxelvoxel--levellevelP(c P(c ≥≥ 1 | n 1 | n ≥≥ 0, t 0, t ≥≥ 4.37) = 0.048 4.37) = 0.048 (corrected)(corrected)P(P(tt ≥≥ 4.37) = 1 4.37) = 1 -- ΦΦ4.374.37 < 0.001 < 0.001 (uncorrected)(uncorrected)

t=4.37

SPM results...SPM results...SPM results...


fMRI :Activations significant atvoxel andcluster level









Conclusions: FWER vs FDRConclusions: FWER Conclusions: FWER vsvs FDRFDR

• Must account for multiplicity– Otherwise have a “fishing expedition”

• FWER– Very specific, less sensitive

• FDR– Less specific, more sensitive– Trouble with cluster inference…

•• Must account for multiplicityMust account for multiplicity–– Otherwise have a Otherwise have a ““fishing expeditionfishing expedition””

•• FWERFWER–– Very specific, Very specific, lessless sensitivesensitive

•• FDRFDR–– Less specific, more sensitiveLess specific, more sensitive–– Trouble with cluster inferenceTrouble with cluster inference……

More Power to Ya!More Power to More Power to YaYa!!Statistical Power• the probability of rejecting the null hypothesis when it is

actually false• “if there’s an effect, how likely are you to find it”?

Effect size• bigger effects, more power

• e.g., MT localizer (moving rings - stationary runs) -- 1 run is usually enough

• looking for activation during imagined motion might require many more runs

Sample size• larger n, more power

• more subjects - longer runs - more runs

Signal:Noise Ratio• better SNR, more power

• stronger, cleaner magnet - more focal coil - fewer artifacts - more filtering

Documents

DISCOS spm mcp - Coma Science Group webpage. Phillips, Centre de Recherches du Cyclotron, ULg, Belgium Multiple Multiple comparison problemcomparison problem DISCOS SPM course, CRC,