Experimental Design

John VanMeter, Ph.D.Center for Functional and Molecular ImagingGeorgetown University Medical Center

Development of an fMRI Experiment

Independent and Dependent VariablesIndependent variables are the parameters that are controlled by the experimenterDependent variables are the data measured by the experimentOne or more independent variables is manipulated in an experiment the effect of which will be measured by the dependent variablesIn most fMRI studies the dependent variable is the change in BOLD fMRI signal

Types of ConditionsTwo basic types of conditions are used in fMRI: Experimental condition is the condition or task of interestControl condition is the task that is subtracted from the experimental conditionRecall that BOLD contrast is non-quantitative

Possible Control Conditions for a Face Processing Study

Confounding FactorsControl condition should in general match the experimental condition as much as possibleConfounding factor is any parameter that varies with the independent variableSelection of a good control condition is important to getting meaningful results

Alcohol ExampleSuppose one found that there was a decrease in fMRI activation for a motor task when subjects drank alcohol as opposed to waterPossible conclusion is that alcohol reduces neuronal activityHowever, should consider other possibilities such as whether the effect of alcohol caused these subject to perform the motor task at the wrong times or less frequently

Subtraction MethodBasic analysis is based on comparing fMRI signal between two conditionsAssumption is that cognitive process of interest is the only difference between the two conditionsPetersen, et al., 1988

Pure Insertion AssumptionInsertion of a single cognitive process does not affect any other processesInteractions between two cognitive processes would invalidate subtraction analysisViolation of Pure Insertion would mean results uninterruptible

Example of Failure of Pure Insertion AssumptionComparison of semantic and letter judgment tasks using three different modalities: mouse, vocal, and covert (silent/mental)Interaction between modality and task in left inferior prefrontal cortexCannot distinguish whether change due to modality or taskJennings, et al., 1997

Analysis and Pure Insertion AssumptionSubtraction analysis assumes pure insertion holds - baseline/control task does not engage any other processesExampleSubtraction of word naming from verb generationWord naming does not require semantic processesWhat if this control condition automatically engages these processes anyway

Main Design ModelsCommon BaselineParallel ComparisonsTailored BaselinesHierarchicalParametricSelective AttentionAdaptation

Common BaselineComparison of two experimental conditions to same controlEx A > CtrlEx B > CtrlDetects areas common to both conditionsAssumes both experimental conditions have similar psychometric properties (ie, task difficulty, equivalent degree of activation across subjects)

Parallel ComparisonsCompare both experimental tasks to each other (seeing vs hearing words)Ex A > Ex BEx B > Ex ACompliments Common BaselineAssumes similar psychometric properties in both A and B

Tailored BaselineUse different control tasks unique to each experimental conditionEx A > Ctrl AEx B > Ctrl BExample: visual display of words vs. false font texthearing words vs.reverse speechAssumes each control task equally removes modality specificsAssumes similar psychometric properties for all conditions - unlikely in most cases Good idea to include a common baseline

Hierarchical SubtractionThree or more task conditions that progressively include additional factorsEx A > RestEx B > Ex AEx C > Ex BExample:Ex A = see words, no responseEx B = repeat words verballyEx C = generate verb associated with wordPure Insertion must hold at all levels

ParametricIncreasing level of difficulty or intensity of taskVariation along a single dimensionA > A > A > AExample - working memory loadUseful for determining function in addition to whereAssumes Pure Modulation - Different levels produce quantitative differences in level of engagementMust be able to define magnitude of differences across levels

Variation of Rate of Extension and Flexion of WristStep function fixed increase in activity irrespective of tapping rateLinear function linear increase in activity with tapping rateVanMeter, et al., 1995

Differential Response

Premotor Primary Motor (M1)

Selective AttentionPresent same stimuli in all conditions but instruct subject to attend to different featuresA B CA B CA B CCan be done implicitly or explicitlyAssumes cognitive process is modified by what is attended toAssumes variables of interest are modulated by selective attentionAssumes passive processing of unattended features does not include cognitive processes of attended feature

Selective Attention: Visual ProcessingCorbetta, et al. presented squares, circles, and triangles that changed in color and moved On each trial all three parameters were variedBy instructing subjects to attend to different features able to identify areas that respond uniquely to shape, color, and motion

Trial 1

Trial 2

Trial 3

Selective Visual Attention ResultsDirected attention to specific features elicited selective activation in corresponding form, color, motion centersAttention to motion -> V5/MTAttention to color -> V2Attention to shape -> V1

Adaptation/Repetition SuppressionRepetitive presentation of same stimulus that produces change in level of activity (typically decreased)Inference is that areas with diminished response are sensitive to stimulus featuresAlso used to diminish response using one type of stimulus to identify response to a novel stimulusPure Modulation Assumption - specific features of stimuli that produce reduction are qualitatively the same

AdaptationSelectivityInvariance for B Stimuli between A & B Stimuli

Adaptation in Visual CortexRebound Index = (% signal change per condition) / (% signal change for identical stimuli)Altmann et al., 2003

Main fMRI Designs for Task PresentationBlock DesignMultiple trials of the same condition are presented consecutivelySwitch back and forth between blocks of experimental and control conditionsEvent RelatedTrials are presented separately and in random order with respect to experimental and control conditions

Reasons for Using Block or Event Related DesignsBlock DesignsBetter at detecting differences between conditions (detection)Some experimental factors take time to occur (e.g. vigilance or sustained attention)Event Related DesignsBetter at detecting differences in HRF (estimation)Some experimental factors are transient or infrequent events by nature (e.g. oddball or n-back tasks)

Considerations for Block DesignsAlternating between experimental and control conditions has limitations (e.g. noun vs verb reading)Generally good idea to include null-task blocks - blocks where subjects do nothing; fixation on a cross preferred to nothingConsider including a progression of blocks in which additional factors are added

Analysis of Block DesignsSubtraction of two conditions only statistical analysis possible of block designs*Thus, baseline/ control events equal in importance to experimental conditionLengths of block types should be equal

Block Length and FrequencyShort block lengths presented close together can limit return to baseline of HRFLonger blocks maximize difference in signal between conditionsBest to use many blocks to minimize noise aliased at frequency of task presentationFrequency of task should be relatively high to minimize low frequency noise such as scanner drift

Superposition and Block Design Indifference to HRF

Event Related (ER) DesignsTrials (aka events) are presented briefly in a random orderISI (interstimulus interval) is the separation between events and is also randomized

Analysis of ER DesignsAverage fMRI signal across all of the presentations of the same event type beginning from onset time of the eventSimilar to ERP (event-related potential) analysis used in analysis of EEG data

Comparison of Block and ER Designs - Detection

ER Designs - Estimation

Principles of ER DesignsBoynton (1996) showed that amplitude and timing of hemodynamic response depends on both intensity and duration of stimulusDale and Buckner (1997) showed that it was possible to extract hemodynamic response function of two different events presented only 1-2 seconds apart

Overlap - Rapid ERDifference in degree of activity due to reduced number of events as run length was kept constant

OverlapOverlap of events possible due to jitterJitter is the randomization of ISI between eventsWithout jitter the 1-2 sec ISI will become equivalent to block design

ER Design AdvantagesFlexibility in designNot every experiment can be turned into block designFlexibility in analysis as same event type can be treated differentlyTrial sorting - choosing events to use in an analysis based on some other parameter such as correctness or reaction time

Semirandom DesignSlight reduction in detection powerBut major increase in estimation efficiency

Mixed DesignsUses a block-design presentationMixAnalysis is done using trial sorting (e.g. examining only trials with correct response)Within a block presented more than one event type

Mixed Design Example - Alzheimers DiseaseTwo separate runs performedRun1 (Encoding)single words nouns presentedinstructed to identify if animate or inanimateRun2 (Retrieval)8 minutes later present nouns; half old half newinstructed to identify old vs new wordsAnalysis examined words in Run1 based on whether they were correctly remembered in Run2

Mixed Design Example - Alzheimer StudyRemembered Trials > Forgotten Trialsin the Encoding runVanMeter, et al. in preparation

Activations and DeactivationsDeactivation - decrease in hemodynamic response in task condition relative to control condition

Good Practices for Experimental DesignSimple methods for reducing confounding factors:Randomization: randomize the order in which conditions presentedCould also be applied to experimenters; dont have one person run all subjects from one group and a second person run all subjects from the other groupCounterbalancing: switch the order in which conditions are presented across subjectsStudy with subjects assigned to one of two groups; try to ensure equal number of men and women in each group in case there are gender effectsRandomize order of runs across subjects; limits practice and order effects

Questions to Ask When Designing an Experiment

Good Practices of fMRI Experimental DesignEvoke the cognitive or other process of interestCollect as much (fMRI) data as possibleCollect data on as many subjects as possibleChoose stimulus and timing to create maximal change in cognitive process of interestTime stimuli presentation of different conditions to minimize overlap in signalUse software to optimize design efficiency for ER designsGet measure of subject behavior in the scanner (ideally related to task)

Put Thought into Experimental DesignAvoid simple comparison of two conditions with minimal thought of what cognitive processes are being comparedDiscussion section of these types of papers come up with a post-hoc just so story as to the meaning of resultsIdeally want to test some modelHave hypotheses that can be confirmed or repudiated

Example of Misuse of fMRI:This is Your Brain on PoliticsNY Times Op-EdIacoboni, et al. wrote an Op-Ed piece (Nov. 2007) on an experiment designed to watch the brains of a group of swing voters as they responded to the leading presidential candidates Never published results in any journalExperimental design consisted of showing 20 subjects (1/2 male & 1/2 female) still photos and videos of speeches from candidates running for presidency at the timeCompared brain activity with response to questionnaires outside scanner

Clintons ResultsVoters who had unfavorable opinions about Sen. Clinton had strong activation of ACCTherefore an emotional center of the brain that is aroused when a person feels compelled to act in two different ways but must choose one. It looked as if they were battling unacknowledged impulses to like Mrs. Clinton.

Male vs Female Response to Clinton and GiulianiMen show little interest in Mrs. Clinton initially but after watching her video they react positively. Women respond to her strongly at first, but their interest wanes after they watch her video. With Mr. Giuliani, the reactions are reversed. Men respond strongly to his initial still photos, but this fades after they see his video. Women grow more engaged after watching his video.For men, Mrs. Clinton is a pleasant surprise. For women, Mr. Giuliani has unexpected appeal.

Obama and McCainBarack Obama and John McCain have work to do. The scans taken while subjects viewed the first set of photos and the videos of Mr. McCain and Mr. Obama indicated a notable lack of any powerful reactions, positive or negative.

Is fMRI Simply Correlational or an Epiphenomenon?Epiphenomenon - a secondary effect or consequence not directly related to the process of interestA major critique of fMRI - we can not state that hemodynamic changes are directly related to neuronal activity given that we dont fully understand the relationship between the two

Reasons why fMRI is not an Epiphenomenon All experiments examine the effect manipulation of the experimental variable has on the dependent variableSame critique could be applied to most types of studies (e.g. drug studies)BOLD contrast has been consistently shown to be a reliable predictor of neuronal activityMost research also relies on convergent evidence such as what is known from animal studies, other imaging techniques (e.g. MEG, EEG), or deficits that arise from a stroke or tumor Logothetis studies demonstrating relationship between BOLD and neuronal activity

**Define mental process to examineDefine tasks to manipulate processMeasure activation during task w/ fMRICompare fMRI signal between tasks****************Within motor cortex there are two different responses to this type of task, demonstrating a specialization in different parts of the motor system****************Blocks of 1s stimuiShows how block design with larger and large number of stimuli is indifferent to simulated different HRF shapes***Word stem completion taskMore activity in block but similar overall pattern*Same task - shows ability of ER to detect latency differences**Randomized stimuli presentation to L & R hemi-fields. 1) Efficiency = number of trials and 2) overlap of different trial types can be useful to getting more trials in the same span of timeDale & Buckner, 1997. **Figure shows ability of ER design to separate out activity in different phases of an experiment - cues and targets****************