Quality Assurance Quality Assurance Quality Assurance Quality Assurance

NITRC Enhancement Grantee MeetingNITRC Enhancement Grantee Meeting

June 18, 2009 June 18, 2009

NITRC Enhancement Grantee MeetingNITRC Enhancement Grantee Meeting

June 18, 2009 June 18, 2009

Susan Whitfield-Gabrieli & Satrajit Ghosh RapidArt

MIT

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

THANKS!THANKS!

Collaborators:Collaborators:

• Alfonso Nieto CastañónAlfonso Nieto Castañón

• Shay MozesShay Mozes

Data:Data:

• Stanford, Yale, MGH, CMU, MITStanford, Yale, MGH, CMU, MIT

Funding: Funding: • R03 EB008673: R03 EB008673: PIs: Satrajit Ghosh, Susan Whitfield-Gabrieli, PIs: Satrajit Ghosh, Susan Whitfield-Gabrieli,

MITMIT

THANKS!THANKS!

Collaborators:Collaborators:

• Alfonso Nieto CastañónAlfonso Nieto Castañón

• Shay MozesShay Mozes

Data:Data:

• Stanford, Yale, MGH, CMU, MITStanford, Yale, MGH, CMU, MIT

Funding: Funding: • R03 EB008673: R03 EB008673: PIs: Satrajit Ghosh, Susan Whitfield-Gabrieli, PIs: Satrajit Ghosh, Susan Whitfield-Gabrieli,

MITMIT

fMRI QA fMRI QA fMRI QA fMRI QA

• Data inspection as well as artifact detection and Data inspection as well as artifact detection and rejection routines are essential steps to ensure rejection routines are essential steps to ensure valid imaging results.valid imaging results.

• ApparentApparent small small differences in data processing differences in data processing may yield may yield largelarge differences in results differences in results

• Data inspection as well as artifact detection and Data inspection as well as artifact detection and rejection routines are essential steps to ensure rejection routines are essential steps to ensure valid imaging results.valid imaging results.

• ApparentApparent small small differences in data processing differences in data processing may yield may yield largelarge differences in results differences in results

QA in fMRIQA in fMRIQA in fMRIQA in fMRI

Before Quality AssuranceBefore Quality Assurance

QA in fMRIQA in fMRIQA in fMRIQA in fMRI

Before QABefore QA

After QAAfter QA

QA: OutlineQA: OutlineQA: OutlineQA: Outline

• fMRI quality assurance protocolfMRI quality assurance protocol

• QA (bottom up)QA (bottom up)

• QA (top down)QA (top down)

• fMRI quality assurance protocolfMRI quality assurance protocol

• QA (bottom up)QA (bottom up)

• QA (top down)QA (top down)

PreprocessingArtifact

Detection

Review DataCheck behaviorCreate mean functional imageReview time series, movie

Interpolate prior to preprocessing

Quality Assurance: PreprocessingQuality Assurance: PreprocessingQuality Assurance: PreprocessingQuality Assurance: Preprocessing

RawImages

Bottom Up: review data

GLMArtifactCheck

- Check registration - Check motion parameters - Generate design matrix template - Check for stimulus corr motion - Check global signal corr with task - Review power spectra - Detect outliers in time series, motion: determine scans to omit /interp or deweight

Quality Assurance: PostQuality Assurance: Post PreprocessingPreprocessingQuality Assurance: PostQuality Assurance: Post PreprocessingPreprocessing

ArtifactCheck

Review Statisitcs Mask/ResMS/RPV Beta/Con/Tmap

Data Review - time series - movie

ArtifactCheck

RFXPreProc

Top Down: review statsBottom Up: review functional images

Data ReviewData ReviewData ReviewData Review

Thresholds

Data ExplorationOutliers

Globalmean

RealignParam

Deviation From meanOver time

MOTIONOUTLIERS

INTENSITYOUTLIERS

COMBINEDOUTLIERS

Including motion parameters as covariatesIncluding motion parameters as covariatesIncluding motion parameters as covariatesIncluding motion parameters as covariates

1. Eliminates (to first order) all motion related residual variance.2. If motion is correlated with the task, this will remove your task activation.3. Check SCM: If there exists between group differences in SCM, AnCova

Power Spectra: HPF Cutoff SelectionPower Spectra: HPF Cutoff SelectionPower Spectra: HPF Cutoff SelectionPower Spectra: HPF Cutoff Selection

.01 .02

Artifact DetectionArtifact DetectionArtifact DetectionArtifact Detection

Scan 79

Scan 95

Artifact Detection/RejectionArtifact Detection/RejectionArtifact Detection/RejectionArtifact Detection/Rejection

Artifact Sources:Artifact Sources: Head motion *Head motion * Physiological : respiration and cardiac effects Physiological : respiration and cardiac effects Scanner noiseScanner noise

Solutions:Solutions: Review dataReview data Apply artifact detection routinesApply artifact detection routines Omit*Omit*, interpolate or deweight outliers, interpolate or deweight outliers

**Include a single regressor for each scan you want to remove, with a 1 Include a single regressor for each scan you want to remove, with a 1 for the scan you want to remove, and zeros elsewhere. for the scan you want to remove, and zeros elsewhere.

*Note # of scan omissions per condition and between groups*Note # of scan omissions per condition and between groupsCorrect analysis for possible confounding effects: Correct analysis for possible confounding effects: AnCova : use # outliers as a within subject covariateAnCova : use # outliers as a within subject covariate

Artifact Sources:Artifact Sources: Head motion *Head motion * Physiological : respiration and cardiac effects Physiological : respiration and cardiac effects Scanner noiseScanner noise

Solutions:Solutions: Review dataReview data Apply artifact detection routinesApply artifact detection routines Omit*Omit*, interpolate or deweight outliers, interpolate or deweight outliers

**Include a single regressor for each scan you want to remove, with a 1 Include a single regressor for each scan you want to remove, with a 1 for the scan you want to remove, and zeros elsewhere. for the scan you want to remove, and zeros elsewhere.

*Note # of scan omissions per condition and between groups*Note # of scan omissions per condition and between groupsCorrect analysis for possible confounding effects: Correct analysis for possible confounding effects: AnCova : use # outliers as a within subject covariateAnCova : use # outliers as a within subject covariate

BOTTOM UPAUDITORY RHYMING > REST

BOTTOM UPAUDITORY RHYMING > REST

T mapResMS

Outlier Scans

ResMS

Before ART

After ART

T map

““TOP DOWN” 2TOP DOWN” 2ndnd level, RFX level, RFX““TOP DOWN” 2TOP DOWN” 2ndnd level, RFX level, RFX

Group Stats ( N = 50 ) Group Stats ( N = 50 ) Group Stats ( N = 50 ) Group Stats ( N = 50 )

Working Memory TaskWorking Memory Task Working Memory TaskWorking Memory Task

Not an obvious problem:Frontal and parietalactivation for a working memory task.

Group Stats (N=50)Group Stats (N=50) 2B Working Memory Task 2B Working Memory Task

Group Stats (N=50)Group Stats (N=50) 2B Working Memory Task 2B Working Memory Task

Find Offending Subjects: 2 of 50 subjectsFind Offending Subjects: 2 of 50 subjectsFind Offending Subjects: 2 of 50 subjectsFind Offending Subjects: 2 of 50 subjects

Artifacts in outlier imagesArtifacts in outlier imagesArtifacts in outlier imagesArtifacts in outlier images

Scan 79

Scan 83

Scan 86

Scan 95

Comparison of Group Stats:Comparison of Group Stats:Working Memory Working Memory (2B>X)(2B>X)

Comparison of Group Stats:Comparison of Group Stats:Working Memory Working Memory (2B>X)(2B>X)

ORIGINAL

FINAL

Comparison of Group Statistics:Comparison of Group Statistics: Default Network Default Network

Comparison of Group Statistics:Comparison of Group Statistics: Default Network Default Network

Method Validation ExperimentMethod Validation Experiment

• Data analyzed: 312 subjects, 3 sessions per subject• Outlier detection based on global signal and movement

• Normality: tests on the scan-to-scan change in global BOLD signal after regressing out the task and motion parameters. Normally-distributed Normally-distributed residuals is a basic assumption of the general linear model. Departures from residuals is a basic assumption of the general linear model. Departures from normality would affect the validity of our analysesnormality would affect the validity of our analyses (resulting p- values could (resulting p- values could not be trusted) not be trusted) If all is well, we should expect this global BOLD signal change to be normally distributed because: average of many sources (central limit theorem )

• Power: the probability of finding a significant effect if one truly exists. Here it represents the probability of finding a significant (at a level of p<.001 uncorrected) activation at any given voxel if in fact the voxel is being modulated by the task (by an amount of 1% percent signal change).

• Data analyzed: 312 subjects, 3 sessions per subject• Outlier detection based on global signal and movement

• Normality: tests on the scan-to-scan change in global BOLD signal after regressing out the task and motion parameters. Normally-distributed Normally-distributed residuals is a basic assumption of the general linear model. Departures from residuals is a basic assumption of the general linear model. Departures from normality would affect the validity of our analysesnormality would affect the validity of our analyses (resulting p- values could (resulting p- values could not be trusted) not be trusted) If all is well, we should expect this global BOLD signal change to be normally distributed because: average of many sources (central limit theorem )

• Power: the probability of finding a significant effect if one truly exists. Here it represents the probability of finding a significant (at a level of p<.001 uncorrected) activation at any given voxel if in fact the voxel is being modulated by the task (by an amount of 1% percent signal change).

• Global signal is not normally distributedIn 48% of the sessions the scan-to-scan change in average BOLD signal is not normally distributed.

This percentage drops to 4% when removing an average of 8 scans per session (those with z score threshold = 3)

• Global signal is not normally distributedIn 48% of the sessions the scan-to-scan change in average BOLD signal is not normally distributed.

This percentage drops to 4% when removing an average of 8 scans per session (those with z score threshold = 3)

Outlier ExperimentOutlier Experiment

• Plot shows the average power to detect a task effect (effect size = 1% percent Plot shows the average power to detect a task effect (effect size = 1% percent signal change, alpha = .001)signal change, alpha = .001)

• Before outlier removal the power is .29 ( 29% chance of finding a significant effect at any of these voxels) After removing an average of 8 scans per session (based on global signal threshold z=3) power improves above .70

• Plot shows the average power to detect a task effect (effect size = 1% percent Plot shows the average power to detect a task effect (effect size = 1% percent signal change, alpha = .001)signal change, alpha = .001)

• Before outlier removal the power is .29 ( 29% chance of finding a significant effect at any of these voxels) After removing an average of 8 scans per session (based on global signal threshold z=3) power improves above .70

Removing outliers improves the power Removing outliers improves the power

THANKS!THANKS!THANKS!THANKS!

Dissemination (NITRC)Dissemination (NITRC)

- International visiting fMRI fellowships @ MGH- International visiting fMRI fellowships @ MGH

- 2 week MMSC @ MGH- 2 week MMSC @ MGH

- SPM8 Courses (local/remote)- SPM8 Courses (local/remote)

-Visiting programs at MIT-Visiting programs at MIT

DocumentationDocumentation

• Manuals, Demos, TutorialsManuals, Demos, Tutorials

• ScriptsScripts

Dissemination (NITRC)Dissemination (NITRC)

- International visiting fMRI fellowships @ MGH- International visiting fMRI fellowships @ MGH

- 2 week MMSC @ MGH- 2 week MMSC @ MGH

- SPM8 Courses (local/remote)- SPM8 Courses (local/remote)

-Visiting programs at MIT-Visiting programs at MIT

DocumentationDocumentation

• Manuals, Demos, TutorialsManuals, Demos, Tutorials

• ScriptsScripts